NOT taking the (canonical) hint: How to estimate a low-quality indexing problem by page type using Google Analytics, Search Console, SEMrush, and Advanced Query Operators

August 5, 2019 By Glenn Gabe Leave a Comment

During a recent crawl analysis and audit of a large-scale site that was negatively impacted by Google’s core updates, I surfaced an interesting SEO problem. I found many thinner and low-quality pages that were being canonicalized to other stronger pages, but the pages didn’t contain equivalent content. As soon as I saw that, I had a feeling what I would see next… since it’s something I have witnessed a number of times before.

Many of the lower-quality pages that were being canonicalized were actually being indexed. Google was simply ignoring rel canonical since the pages didn’t contain equivalent content. That can absolutely happen and I documented that in a case study a few years ago. And when that happens on a large-scale site, thousands of lower-quality pages can get indexed (without the site owner even knowing that’s happening).

For example, imagine a problematic page type that might account for 50K, 100K, or more pages indexed. And when Google takes every page indexed into account when evaluating quality, you can have a big problem on your hands.

In the screenshot below, you can see that Google was ignoring the user-declared canonical and selecting the inspected url instead. Not good:

In addition to just getting indexed, those lower-quality pages might even be ranking in the search results for queries and users could be visiting those pages by mistake. Imagine they are thin, lower-quality pages that can’t meet or exceed user expectations. Or maybe they are ranking instead of the pages you intend to rank for those queries. In a case like that, the problematic pages are the ones winning in the SERPs for some reason, which is leading to a poor user experience. That’s a double whammy SEO-wise.

Quickly Estimating The Severity Of The Problem
After uncovering the problem I mentioned above, I wanted to quickly gauge how bad of a situation my client was facing. To do that, I wanted to estimate the number of problematic pages indexed, including how many were ranking and driving traffic from Google. This would help build a case for handling the issue sooner than later.

Unfortunately, the problematic pages weren’t all in one directory so I needed to get creative in order to drill into that data (via filtering, regex, etc.) This can be the case when the urls contain certain parameters or patterns of characters like numerical sequences or some other identifying pattern.

In this post, I’ll cover a process I used for roughly discovering how big of an indexing problem a site had with problematic page types (even when it’s hard to isolate the page type by directory). The process will also reveal how many pages are currently ranking and driving traffic from Google organic. By the end, you’ll have enough data to tell an interesting SEO story, which can help make your case for prioritizing the problem.

A Note About Rel Canonical – IT’S A HINT, NOT A DIRECTIVE
For any site owner that’s mass-canonicalizing lower-quality pages to non-equivalent pages, then this section of my post is extremely important. For example, if you think you have a similar situation to what I mentioned earlier and you’re saying, “we’re fine since we’re handling the lower-quality pages via rel canonical…”, then I’ve got some potentially bad news for you.

As mentioned earlier, rel canonical is just a hint, and not a directive. Google can, and will, ignore rel canonical if the pages don’t contain equivalent content (or extremely similar content). Again, I wrote a case study about that exact situation where Google was simply ignoring rel canonical and indexing many of those pages. Google’s John Mueller has explained this many times as well during webmaster hangout videos and on Twitter.

And if Google is ignoring rel canonical on a large-scale site, then you can absolutely run into a situation where many lower-quality or thin pages are getting indexed. And remember, Google takes all pages indexed into account when evaluating quality on a site. Therefore, don’t just blindly rely on rel canonical. It might not work out well for you.

Walking through an example (based on a real-world situation I just dealt with):
To quickly summarize the situation I surfaced recently, there are tens of thousands of pages being canonicalized to other pages on the site that aren’t equivalent content-wise. Many were being indexed since Google was ignoring rel canonical. Unfortunately, the pages weren’t located in a specific directory, so it was hard to isolate them without getting creative. The urls did contain a pattern, which I’ll cover soon.

My goal was to estimate how many pages were indexed and how many were ranking and driving traffic from Google organic. Remember, just finding pages ranking and driving traffic isn’t enough, since there could be many pages indexed that aren’t ranking in the SERPs. Those are still problematic from an SEO standpoint.

The data would help build a case for prioritizing the situation (so my client could fix the problem sooner than later). It’s a large-scale with many moving parts, so it’s not like you can just take action without making a strong case. Volatility-wise, the site was impacted by a recent core update and there could be thousands (or more) lower-quality or thin pages indexed that shouldn’t be.

With that out of the way, let’s dig in.

Gauging The Situation & The Limits Of GSC
To gauge the situation, it’s important to understand how big of a problem there is currently and then form a plan of attack for properly tackling the issue. In order to do this, we’ll need to rely on several tools and methods. If you have a smaller site, you can get away with just using Google Search Console (GSC) and Google Analytics (GA). But for larger-scale sites, you might need to get creative in order to surface the data. I’ll explain more about why in the following sections.

Index Coverage in GSC – The Diet Coke of indexing data.
The index coverage reporting in GSC is awesome and enables you to view a number of important reports directly from Google. You can view errors, warnings, pages indexed, and then a list of reports covering pages that are being excluded from indexing. You can often find glaring issues in those reports based on Google crawling your site.

Based on what we’re trying to surface, you might think you can go directly to the Valid (and Indexed) report, export all pages indexed, then filter by problematic page type, and that would do the trick. Well, if you have a site with less than 1,000 pages indexed, you’re in luck. You can do just that. But if you have more than 1,000 pages, then you’re out of luck.

GSC’s Index Coverage reporting only provides 1,000 urls per report and there’s no API (yet) for bulk exporting data. Needless to say, this is extremely limiting for large-scale sites. To quote Dr. Evil, it’s like the Diet Coke of indexing data. Just one calorie… not thorough enough.

Search Console API & Analytics Edge
Even though exporting urls from the Valid (and Indexed) category of index coverage isn’t sufficient for larger-scale sites, you can still tap into the Search Console API to bulk export Search Analytics data. That will enable you to export all landing pages from organic search over the past 16 months that have impressions or clicks (basically pages that were ranking and driving traffic from Google). That’s a good start since if a page is ranking in Google, it must be indexed. We still want data about pages indexed that aren’t ranking, but again, it’s a start.

My favorite tool for bulk exporting data from GSC is Analytics Edge. I’ve written about Analytics Edge multiple times and you should definitely check out those posts to get familiar with the Excel plugin. It’s powerful, quick, reasonably priced, and works like a charm.

For our situation, it would be great to find out how many of those problematic pages are gaining impressions and clicks in Google organic. Since the pages are hard to isolate by directory or site section, we can export all landing pages from GSC and then use Excel to slice and dice the data via filtering. You can also use the Analytics Edge Core Add-in to use regex while you’re in the process of exporting data (all in one shot). More about that soon.

Exporting landing page data from GSC via Analytics Edge

A Note About Regex For Slicing And Dicing The Data
Since the pages aren’t in one directory, using regular expressions (regex) is killer here. Then you can filter using regular expressions that target certain url patterns (like isolating parameters or a sequence of characters). To do this, you can use the Analytics Edge Core Plugin in conjunction with the Search Console connector so you can export the list of urls AND filter by a regular expression all in one macro.

I won’t cover how to do that in this post, since that can be its own post… but I wanted to make sure you understood using regex was possible.

You can also use Data Studio and filter based on regular expressions (if you are exporting GSC data via Data Studio). The core point is that you want to export all pages from GSC that match the problematic page type. That will give you an understanding of any lower-quality pages ranking and driving traffic (that match the page type we are targeting).

Now let’s get more data about landing pages driving traffic from Google organic via Google Analytics.

Google Analytics with Native Regex
In order to find all problematic page types that are driving traffic from Google organic, fire up GA and head to Acquisition, All Traffic, and then Source/Medium. This will list all traffic sources driving traffic to the site in the timeframe you selected. Choose a timeframe that makes sense based on your specific situation. For this example, we’ll select the past six months.

Then click Google/Organic to view all traffic from Google organic search during the timeframe. Now we need to dimension the report by landing page to view all pages receiving traffic from Google organic. Under Primary Dimension, click Other, then Commonly Used, and then select Landing Page. Boom, you will see all landing pages from Google organic.

But remember, we’re trying to isolate problematic page types. This is where the power of regular expressions comes in handy (as mentioned earlier). Unlike GSC, Google Analytics natively supports regex in the advanced search box, so dust off those regex skills and go to town.

Let’s say all of the problematic page types have two sets of five-digit numbers in the url. They aren’t in a specific directory, but both five-digit sequences do show up in all of the urls for the problematic page type separated by a slash. By entering a regular expression that captures that formula, you can filter your report to return just those pages.

For this example, you could use a regex like:
\d{5}.\d{5}

That will capture any url that contains five digits, any character after that (like a slash), and then five more digits. Now all I need to do is export the urls from GA (or just document the number of urls that were returned). Remember, we’re just trying to estimate how many of those pages are indexed, ranking and/or driving traffic from Google. The benefit of exporting is that you can send them through to your dev team so they can further investigate the urls that are getting indexed by mistake.

Filtering landing pages by regex in Google Analytics

Note, you can also use Analytics Edge to bulk export all of your landing pages from Google Analytics (if it’s a large-scale site with tens of thousands (or more) pages in the report. And again, you can combine the Analytics Edge Core Plugin with the GA connector to filter by regex while you are exporting (all in one shot).

Third-party tools like SEMrush
OK, now our case is taking shape. We have the number of pages ranking and driving traffic from Google organic via GSC and GA. Now let’s layer on even more data.

Third-party search visibility tools provide many of the queries and landing pages for each domain that are ranking in Google organic. It’s another great data source for finding pages indexed (since if the pages are ranking, they must be indexed).

You can also surface problematic pages that are ranking well, which can bolster your case. Imagine a thin and/or lower-quality page ranking at the top of the search results for a query, when another page on your site should be there instead. Examples like this can drive change quickly internally. And you can also see rankings over time to isolate when those pages started ranking, which can be helpful when conveying the situation to your dev team, marketing team, CMO, etc.

For example, here’s a query where several pages from the same site are competing in the SERPs. You would definitely want to know this, especially if some of those urls were lower-quality and shouldn’t be indexed. You can also view the change in position during specific times.

Viewing lower-quality pages ranking in the SERPs via SEMrush

For this example, we’ll use one of my favorite SEO tools, SEMrush. Once you fire up SEMrush, just type in the domain name and head to the Organic Research section. Once there, click the Pages report and you’ll see all of the pages that are ranking in Google from that domain (that SEMrush has picked up).

Note, you can only export a limited set of pages based on your account level unless you purchase a custom report. For example, I can export up to 30K urls per report. That may be sufficient for some sites, while other larger-scale sites might need more data. Regardless, you’ll be gaining additional data to play with including the number of pages ranking in Google for documentation purposes (which is really what we want at this stage).

You can also filter urls directly in SEMrush to cut down the number of pages to export, but you can’t use regex in the tool itself. Once you export the landing pages, you can slice and dice in Excel or other tools to isolate the problematic page type.

Query Recipes – Hunting down rough indexing levels via advanced search operators
OK, now we know the number of pages indexed by understanding how many pages are ranking or receiving traffic from Google. But that doesn’t tell us the number of pages indexed that aren’t ranking or driving traffic. Remember, Google takes every page indexed into account when evaluating quality, so it’s important to understand that number.

Advanced query operators can be powerful for roughly surfacing the number of pages indexed that match certain criteria. Depending on your situation, you can use a number of advanced search query operators together to gauge the number of pages indexed. For example, you can create a “query recipe” that surfaces specific types of pages that are indexed.

It’s important to understand that site commands are not perfectly accurate… so you are just trying to get a rough number of the pages indexed by page type. I’ve found advanced search queries like this very helpful when digging into an indexing problem.

So, you might combine a site command with an inurl command to surface pages with a certain parameter or character sequences that are indexed. Or maybe you combine that with an intitle command to include only pages with a certain word or phrase in the title. And you can even combine all of that with text in quotes if you know a page type contains a heading or text segment in the page content. You can definitely get creative here.

If you repeat this process to surface more urls that match a problematic page type, then you can get a rough number of pages indexed. You can’t export the data, but you can get a rough number to add to your total. Again, you are building a case. You don’t need every bit of data.

Here are some examples of what you can do with advanced query operators:

Site command + inurl:
site:domain.com inurl:12/2017
site:domain.com inurl:pid=

Site command + inurl + intitle
site:domain.com inurl:a5000 intitle:archive
site:domain.com inurl:tbio intitle:author

Site command + inurl + intitle + text in quotes
site:domain.com inurl:c700 intitle:archive “celebrity news”

Using advanced query operators enable you to gain a rough estimate of the number of pages. You can jot down the number of pages returned for each query as you run multiple searches. Note, you might need to run several advanced queries to hunt down problematic page types across a site. It can be a bit time-consuming, and you might get flagged by Google a few times (by being put in a “search timeout”), but they can be helpful:

Using advanced query operators to hunt down low-quality pages that are indexed.

Summary – Using The Data To Build Your Case
We started by surfacing a problematic page type that was supposed to be canonicalized to other pages, but was being indexed instead (since the pages didn’t contain equivalent content). Google just wasn’t taking the hint. So, we decided to hunt down that page type to estimate how many of those urls were indexed to make a case for prioritizing the problem.

Between GSC, GA, SEMrush, and advanced query operators, we can roughly understand the number of pages that are indexed, while also knowing if some are ranking well in Google and driving traffic. In the real-world case I just worked on, we found over 35K pages that were lower-quality and indexed. Now my client is addressing the situation.

By collecting the necessary data (even if some of it is rough), you can tell a compelling story about how a certain page type could be impacting a site quality-wise. Then it’s important to address that situation correctly over the long-term.

I’m sure there are several other ways and tools to help with understanding an indexing problem, but this process has worked well for me (especially when you want to quickly estimate the numbers). So, if you ever run into a similar situation, I hope you find this process helpful. Remember, rel canonical is just a hint… and Google can make its own decisions. And that can lead to some interesting situations SEO-wise. It’s important to keep that in mind.

GG

Google’s Core Algorithm Updates and The Power of User Studies: How Real Feedback From Real People Can Help Site Owners Surface Website Quality Problems (And More)

July 2, 2019 By Glenn Gabe Leave a Comment

Google just rolled out another broad core algorithm update on June 3 (which was preannounced by Google’s Danny Sullivan.) And once again, the core ranking update was big. It wasn’t long before you could see significant impact from the update across sites, categories, and countries. Some sites surged, while others dropped off a cliff. And that’s par for the course with Google’s core updates.

For example, here are three examples of drops from the June 2019 Google Core Update:

But I’m not here to specifically cover the June update. Instead, I’m here to cover an extremely important topic related to all broad core ranking updates – conducting user studies. It’s something I have mentioned in a number of my posts about major algorithm updates, and Googlers have mentioned it too by the way. More on that soon.

My post today will cover the power of user studies as they relate to core ranking updates, and provide feedback from an actual user study I just conducted for a site impacted by several major updates. By the end of the post, I think you’ll understand the value of a user study, and especially how it ties to Google’s core updates by gaining feedback from real people in your target audience.

Google: Take A Step Back And Get Real Feedback From Real People:
After core updates roll out, Google’s John Mueller is typically pummeled with questions about how to recover, which factors should be addressed to turn things around, etc. And as I’ve documented many times in my posts about core updates, there’s never one smoking gun for sites negatively impacted. Instead, there’s typically a battery of smoking guns. John has explained this point many times over the years and it’s incredibly important to understand.

But beyond just taking a step back and surfacing all potential quality problems, John has explained another important point. He has explained that site owners should gain objective feedback from real users. And I’m not referring to your spouse, children, coworkers, top customers, etc. I’m talking about feedback from objective third parties. i.e. People that don’t know your site, business, or you before visiting the site.

When you conduct a study like that, you can learn amazing things. Sure, some of the feedback will not make you happy and will be hard to take… but that’s the point. Figure out what real people think of your site, the user experience, the ad situation, the content, the writers, etc. and then form a plan of attack for improving the site. It’s tough love for SEO.

Here is one video of John explaining that site owners should gain feedback from objective third-parties (at 13:46 in the video). Note, it’s one of several where John explains this:

Conducting User Studies Through The Lens Of Google’s Core Updates:
When you decide to conduct a user study in order to truly understand how real people feel about a site, it’s important to cover your bases. But it can be a daunting task to sit back and try to craft questions and tasks for people that will capture how they feel about a number of core site aspects. As I explained above, you want to learn how people really feel about your content-quality, the writers, the user experience, the advertising situation, trust-levels with the site, and more. So, crafting the right questions is important.

But where do you even begin??

Well, what if Google itself actually crafted some questions for you? Wouldn’t that make the first user study a lot easier? Well, they have created a list of questions… 23 of them to be exact. And they did that in 2011 when medieval Panda roamed the web.

The list of questions crafted by Amit Singhal in the blog post titled More guidance on building high-quality sites provides a great foundation for your first user study related to Google’s core algorithm updates.

Google’s blog post from 2011 containing 23 Panda questions.

For example, the questions include:

Would you trust the information presented in this article?
Is this article written by an expert or enthusiast who knows the topic well, or is it more shallow in nature?
Would you be comfortable giving your credit card information to this site?
Does the article provide original content or information, original reporting, original research, or original analysis?
Does the page provide substantial value when compared to other pages in the search results?
How much quality control is done on content?
And more…

As you can see, these are incredibly important questions to review. The questions can absolutely help you better understand how real users are experiencing your site, how they feel about your site, and ultimately, the questions can help craft a remediation plan covering what you need to change or improve on your own site.

I have used these questions (or variations of them) to run both quick and dirty user studies, and formal studies. The feedback you can receive is absolutely gold. Not just gold, but SEO gold in the age of broad core ranking updates.

Let’s face it, this is exactly the type of information that Google is trying to evaluate algorithmically. So, although it’s not easy to run user studies, and it can be time-consuming and tedious, it’s one of the most important things you can do as a site owner.

Beyond The 23 Panda Questions, More Ideas From The Quality Rater Guidelines (QRG)
The Panda questions provide a great foundation, but you can absolutely run more user testing using Google’s Quality Rater Guidelines (QRG) as your foundation. And there are a boatload of topics, ideas, and questions sitting in the 166-page guide that Google uses with its own quality raters.

Google’s Quality Rater Guidelines (as of May 2019)

I won’t spend too much time on the QRG in this post, but some topics you might test include:

Page quality.
Needs-met ratings.
Usability.
Monetization and Advertising.
Authors, writers, review boards.
Site reputation.
E-A-T.
User intent.
And more…

Now, you can just trust me (and John) and think that user testing is important, or you might want more information. For example, like seeing examples of what you can really learn from a user study. Well, I’ve got you covered. I just conducted a user study for a site that was heavily impacted by the March core update (and that has seen major volatility during several core updates over the years). The feedback we received from the user study was awesome and I’m going to share some of it with you (without revealing the site). I think you’ll get the power of user studies pretty quickly.

User Testing Results: What You Can Learn From Real People: Health/Medical Case Study
Again, the site has seen big swings (up and down) during various core updates and I’ve been helping them identify all potential quality problems across the site (including content-quality, technical SEO, user experience, advertising situation, site reputation, UX barriers, and more).

After fully auditing the site, I used the Panda questions mentioned earlier as the foundation for the user study and tailored some of those questions for the niche and site. Below, I’ll provide some of things we learned that I thought were extremely important for my client to understand. Remember, this is real feedback from real people.

Test-wise, I not only used multiple choice questions, but I also used open-ended questions to learn more about how each user felt about certain situations. In addition, I used a platform that provides session recordings of each user going through the study. For this study I used usertesting.com and I’ll explain more about testing platforms later in this post.

I can tell you that watching and listening to people experience a site is absolutely fascinating. There is so much you can learn from hearing the reaction of users, picking up things they say, and watching how they navigate a site or page.

So, the combination of quantitative feedback, qualitative feedback, and viewing recorded sessions provides the ultimate recipe for surfacing potential problems on a site. And that feedback can directly help site owners craft a remediation plan that goes beyond fixing minor issues. Instead, you can start to address deeper issues and problems. And that’s exactly what Google’s core updates are about… Google is evaluating a site overall and not just looking at one or two factors. Remember, there’s never one smoking gun.

The combination of quantitative and qualitative feedback,
plus recorded user sessions = SEO Gold.

First, some quick background information about the user study:
By the time I was setting up the test, I had already fully analyzed the site and provided many areas for improvement. But, we wanted to gain feedback from real users in the site’s target audience about a number of important topics. Also, I wanted to use the 23 Panda questions as a foundation for the test.

Audience Selection:
Since usertesting.com has a panel of over one million people, I was able to select specific demographic information that enabled us to make sure the test participants were part of my client’s target audience. For example, I was able to select gender, age, household income, if they were parents (and how old their children were), job status, web expertise, and more. I’ll cover more about this later.

Selecting an audience is powerful within usertesting.com’s test creation process.

So, what were some things I wanted to learn from the participants? Here are a few of the things I was interested in:

Did users trust the information provided in several articles I asked them to read?
Did they think the articles were written by experts, or just people heavily interested in a topic?
Was the content original? Or did they think it could easily be found elsewhere on the web?
Did they recognize the brand? How about the founders and writers?
How did they feel about recency, original publication dates, if the articles were updated, and how that was treated on the page?
I asked them to review and provide feedback about the background and experience of the site owners, authors, and the medical review board.
I wanted to know if the participants thought there was an aggressive, disruptive, or deceptive advertising situation (since this was a problem when I first started analyzing the site).
And more… there were 39 different questions and tasks I had the participants go through.

Below, I’ll cover some pieces of feedback that we thought were extremely helpful. By the way, some of the responses (and video clips) were eye-opening. I’ll provide the details below.

Examples of feedback from the user study (in no specific order):

Balance – Several participants mentioned the importance of balance in the article. For example, thoroughly covering the benefits AND risks of certain topics. Again, this is something that can be very important in articles, especially YMYL articles.
Triggers – I learned that certain words were triggers for some people, which I could only hear in the video clips. I would have never known that from multiple choice questions. For example, when certain words were read aloud, some participants would react in a way that clearly showed how they felt about that topic. They even said, “whenever I read {enter word here}, that immediately throws up a red flag for me”. Wow, amazing feedback for the site owners.
Sources and Credibility – Along the same lines, the sources and citations were extremely important for some of the participants. Some explained that if they see Wikipedia as a source, they immediately become skeptical. One even said it discredits the article. For example, one user said, “wait, so it’s reviewed by a doctor, but it cites Wikipedia… not sure I trust this article at all.”
Trust & Reactions – When asked about if a certain participant trusted one of the articles, she laughed out loud. Again, hearing people in the video is incredibly powerful. And laughing is typically not a good thing for a YMYL site. :)
Publish Dates – There were several important pieces of feedback regarding publish dates, updated dates, etc. First, some assumed that if there was an updated date on the article, then that meant the entire article had been fully reviewed again. That can be deceptive, since the articles just had specific pieces updated.
More About Publish Dates – Some participants absolutely wanted to see the original publish date and the updated date. They did not just want the updated date, since that makes them search for clues about when the article was originally published. Some participants explained the process they go through to find the original publish date, which included checking the sources being cited (and the dates associated with those sources). And then they use a savvy approach of checking the comments for dates.
Social Proof – I heard one participant explain how if she sees a lot of comments, then that means it must be a popular website. Very interesting… Comments are tough for many sites due to the onslaught of spam, the time involved in moderating comments, etc., but they do seem important for some people.
Author Expertise – Several participants wanted to know the background of the writers as they were reading each article. Since the articles they were reading covered health topics, they immediately went into “skeptical mode”. This was important to see and underscores the importance of having experts write the content.
Citing Sources – Several participants explained that just a link to a source wasn’t enough for some articles. They wanted to see stats and facts backing up some claims (in the article itself). For example, maybe providing some of the data directly in the article versus just linking out to another article.
“Just A Blog…” – I heard several remarks comparing blogs to medical websites. For the health niche, this was very interesting feedback. There was a negative stigma with blogs for some users, especially for health/medical topics.
Advertising Situation – Advertising-wise, there were also some interesting pieces of feedback. Remember, there was an aggressive advertising situation when I first started helping the client, so I was extremely interested in hearing what the participants thought of the current ad situation (which has improved, but the site owners haven’t moved as far as I would like them to). I heard one user literally counting the number of ads as she scrolled down the page. 1, 2, 3, wait more, 4, 5. But in a strange twist, she then said the ad situation was fine… She knew there were a number of ads, but didn’t find them distracting. It’s extremely important to make sure the advertising situation is ok, since Google has explained that aggressive ads can impact a site algorithmically over time.
Affiliate Marketing – Regarding affiliate links, I did hear, “Are they just trying to sell me something?? Ok, they probably are…” This is something I have brought up to my client during the audit and it’s a tough conversation to have. But remember, Google has explained that there’s a fine balance when delving into affiliate links or affiliate marketing in general. There must be a lot of value added versus monetization. If the scale tips in the wrong direction, bad things can happen Google-wise. So this piece of feedback was extremely important to see/hear directly from users.
Author Expertise – When asked about the expertise of the author of an article, the user started scrolling to find the author information and then said, “Wait, it’s a blog… no, I don’t trust the author at all.” I heard this type of comment several times during the user study. More about building a brand and credibility soon.
Content-Quality – When asked about original content across the articles, almost all of the users in the study said there was some original content, but some of it could easily be found in other places across the web. Not one person said the content was original. This underscores the importance of tackling subject matter where you can provide original content, ideas, perspectives, etc. If you write about what many others are writing about, the content can be viewed as quasi-original. That’s not good enough for a tough niche.
Content Value – When asked about substantial value from the content compared to other articles on the topic, every one of the users said it was average compared to the others. You clearly don’t want to strive for “average”. You want 10X content. This was great for my client to see. They have strong articles overall, but users saw them as average compared to the competition.
Side note: SERP UX – When watching users go to Google and look for a competing article, it was fascinating to see several scroll right by the featured snippet and select something a little farther down the page (in the standard organic results). Sure, this isn’t a large sample size, but just an interesting side note.
Site Design – When researching other articles on a topic, a user commented that all the sites look the same. And those sites ranged from some of the top health sites on the web to academic sites to health blogs. Site design, branding, etc. comes into play here and it’s something that I don’t think many focus on enough.
Brand Recognition – Regarding brand, every one of the users in the study said they never heard of the site, brand, etc. This is clearly a signal that the site owners need to work on branding. For example, getting the brand out there more via PR, reaching eyeballs beyond their core audience, etc.
Recency – For health topics, I heard a user explain they definitely want to see more recent articles on a topic. The article they were reading was a few years old and that didn’t seem sufficient for her. Recency seemed important (but it must actually be recent and not just an “updated on XX” tag slapped on the page).
Affiliate marketing – More comments about “they are advertising {enter product here}” while reading an article. So yes, users pick up on affiliate links. Again, the value from the article must outweigh the monetization piece.
Citing sources – There were positive comments about certain sources that were cited, like consumer reports, a scientific study, etc. For health articles, I saw users in the video checking the sources at the bottom of the page, which could help build credibility.
Medical review board – Overall, the users liked that articles were reviewed by a medical review board. I heard this several times while reviewing the recorded sessions of participants reading the articles.
Expertise and Credibility – When asked about the expertise and background of the site owners, authors, and medical review board, there were plenty of interesting comments. For example, having a medical review board with various types of doctors, nutritionists, etc. seemed to impress the participants. But I did hear feedback about wanting to see those credentials as quickly as possible on the page. In other words, don’t waste someone’s time. Don’t be too cute. Just provide the most helpful information that builds credibility as quickly as possible.
Awards and Accolades – For various awards won, users want a link to see more information about that (or they wanted to see more on the page itself). It’s clearly not good enough in this day and age to simply say you won something. Let’s face it… anyone can say that. They want proof.
Trust – When asked if they would be comfortable giving their credit card information to the site, most responded, “I’m not sure I would go that far…” or “No, definitely not”. So, there were clearly some breakdowns with trust and credibility. I saw this throughout various responses in the study. My client has some work to do on that front.
UX barriers – I noticed errors pop up twice while reviewing the video clips of users going through the site. If these are legit errors, then that’s extremely helpful and important to see. I passed the screenshots along to my client so their dev team could dig in. It’s just a secondary benefit of user testing (with video recordings of each session).
And there were many more findings…

As you can see, between reading their responses, hearing their reactions, and then watching each video session, we gained a ton of amazing feedback from the user study. Some of the feedback was immediately actionable, while other pieces of feedback will take time to address. But overall, this was an incredible process for my client to go through.

User Testing Platforms – Features & User Panel
If you just read the sample of findings above and are excited to conduct your own user study, you might be wondering where to start. Well, there are several important things to consider when preparing to launch a user study. The first is about the platform you will use.

Usertesting.com is probably the most well-known platform for conducting user studies and it’s the one I used for this test. I was extremely impressed with the platform. The functionality is killer and their panel of over one million people is outstanding.

In addition, participants sign a non-disclosure agreement (NDA), which can help reduce the chance of your test getting shared publicly. Some sites wouldn’t care about this, but others would care. For example, I know a number of my clients would NOT want the world knowing they are running a user study focused on trust, quality, advertising situation, etc.

Audience-wise, I was able to select a range of criteria for building our target audience for the user study (as covered earlier). This enabled me to have participants that were closely tied to my client’s target audience. It’s not perfect, but can really help focus your audience.

Functionality-wise, you can easily create multiple choice questions, open-ended questions, etc. You can also use Balanced Flow to send users through two different test flows. This can enable you to test different paths through a site or different customer experiences.

Here are some screenshots from the test creation process:

Pricing-wise, usertesting.com isn’t cheap… but could be well worth the money for companies that want to perform a number of user tests (across a range of actions). Remember, the sky’s the limit with what you can test. For example, site design, usability, features, content-quality, site trust, and more. I was ultra-impressed with usertesting.com.

Beyond usertesting.com, I also looked into UsabilityHub (Google is a client of theirs btw) and userlytics. I have not used these other platforms, but they could be worth looking into since they also have large panels of users and what seems to be strong features.

Closing Tips and Recommendations:
Before ending this post, I wanted to provide some closing tips and recommendations when setting up your first test. I am by no means an expert on user testing, but I have learned some important lessons while crafting tests:

First, user testing is not easy. It can be time-consuming and tedious (especially when analyzing the results). Build in enough time to craft your questions and flow, and then enough time for fully analyzing the results. You might be surprised how much time it takes to get it right.
For Google’s core updates, you can definitely use the 23 Panda questions as a foundation for your test. You also might take a subset of those questions and then tailor them for a specific niche and site. After that, you can use the Quality Rater Guidelines as a foundation for additional tests.
Try to not ask leading questions. It’s very hard to avoid this… but don’t sway the results by leading someone down a certain response path.
Session recordings are killer. Make sure you watch each video very carefully. I’ve found you can pick up some interesting and important things while watching and listening to users that are trying to accomplish a task (or just while they are reviewing a site).
Take a lot of notes… I had a text editor up and running so I could timestamp important points in the videos. Then it was easy to go back to those clips later on while compiling my results.
Try to gain both quantitative and qualitative feedback from users. Sure, multiple choice questions are great and can be quick and easy, but open-ended questions can yield important findings that might not be top-of-mind when crafting your test. And then layer on videos of each session, and you can gain a solid view of how real users view your site, content, and writers.
Find the right balance for the number of participants. Usertesting.com recommends up to 15 participants for a test. Don’t overload your test, which can lead to data overkill. Try different numbers of participants over a series of tests to see what yields the most valuable results. For some tests, 5 participants might be enough, while other tests might require 15 (or more).

Summary – User testing can be a powerful tool for sites impacted by Google’s core ranking updates
Google has explained many times that it is looking at many factors when it comes to broad core ranking updates. That includes content-quality, technical SEO, user experience (UX), advertising situation, E-A-T, and more. Google’s John Mueller has also explained that it’s important to take a step back and objectively analyze your site.

Well, a great way to objectively analyze your site is by conducting user testing. Then you can have objective third-parties go through your site, content, features, etc., and provide real feedback. I’ve found this process to be extremely valuable when helping companies impacted by major algorithm updates since it can surface qualitative feedback that is hard to receive via other means. I recommend trying this out for your own site (even if you haven’t been impacted by core updates). I think you’ll dig the results. Good luck.

GG

Beyond The 1K Limit – How To Bulk Export Data From GSC By Search Appearance Via Analytics Edge (including How-to, Q&A, and FAQ)

June 11, 2019 By Glenn Gabe Leave a Comment

Google has been releasing new features in the search results more and more recently that can have a big impact on SERP treatment, click-through rate, and potentially traffic. Three of those features are part of Google’s “best answer carousels” and include Q&A, How-to, and FAQ snippets. Q&A has been live for a while already, while How-to and FAQ were just rolled out during Google I/O. Note, you can read my post about How-to snippets to learn more about how they work, what they look like, etc.

I have several clients heavily using these formats, so I’ve been analyzing their performance via Google Search Console (GSC) recently — via the new enhancement reports and the Performance reporting. For example, once you start marking up pages using Q&A, How-to, or FAQ markup, you will see new reports show up under the Enhancements tab in GSC. And those reports can be very helpful for understanding errors, warnings, and valid pages.

How To Analyze Performance In GSC By Search Feature:
From a clicks, impressions, and CTR standpoint, you can check the Performance report to view your data over time. Note, if you have Discover data, then there will be two reports under Performance. The first will say Search Results and the second will be titled Discover.

Once in the Performance report (or the “Performance in search results” report), the Search Appearance tab enables you to drill into data by specific feature. You can see that the site from earlier has both How-to and Q&A results. If you click each category title, then you will be isolating that search feature in the reporting (i.e. the reports will be filtered by that search feature). So, you can view queries, landing pages, etc. for just Q&A or How-to results. This applies to AMP as well.

The 1K Row Limit In GSC. The Bane Of A Site Owner’s Existence
Filtering the reporting by search feature is powerful, but remember, GSC only provides one thousand results by report in the web UI and you can only export those one thousand results. For smaller sites, that should be fine. But for larger-scale sites with thousands, tens of thousands, or more listings, then the reporting can be extremely limiting.

For situations like that, what’s a site owner to do??

Analytics Edge To The Rescue Again. Exporting Beyond 1K Results By Search Feature:
I’ve written several posts about Analytics Edge before, and it’s still my go-to tool for exporting data in bulk from GSC. It’s a powerful Excel plugin that enables you to quickly and efficiently export bulk data from GSC, GA, and more.

Below, I’ll take you step-by-step through exporting your data by Search Appearance from GSC. If you’re a large-scale site that’s using Q&A, How-to, FAQ, and even AMP, then you’re going to dig this. Let’s jump in.

How to use Analytics Edge to bulk export data by Search Appearance:
Note, this will be a two-phase approach. The first run will enable us to pull all Search Appearance codes for the specific property in GSC. Then the second run will enable us to pull all data by that Search Appearance code.

Phase One:

Download and install the Analytics Edge free or core add-in. There’s a free trial for the core add-in if you wanted to simply test it out. But the free add-in will work as well (just with less functionality). After installing the add-in, you should register it in Excel.
Next, install the Search Console connector by clicking the License/Update button in the menu. You can watch this short video to learn how to install connectors. You can click the Google Search row to pull up the connector details (where you can choose to install that connector).
Once you install Analytics Edge and the Search Console connector, access the options in the Analytics Edge menu at the top of Excel. Click the Google Search drop-down and select Accounts. This is where you will connect Analytics Edge with the Google account(s) you want to download data from. Go through the process of connecting the Google account you want to work with. You can also make one account the default, which will save you time in the future.
Once you connect your account, click Google Search in the Connectors section, and then Search Analytics.
Name your Macro and click OK.
Select an account and then a property from GSC.
We will use a two-phase approach for exporting data by Search Appearance. First, we are going to view the various options we have under Search Appearance (there will be codes that show up representing each SERP feature available to a property). We will use these codes during our second run to pull all data for each specific search feature (like How-to, Q&A, FAQ, AMP, etc.)
Under the Fields tab, select searchAppearance, which will move that option to the Dimensions window.
For the Dates tab, you can leave “Last 3 Months” active (which is the default).
Leave everything else the same and click “Finish”.
Analytics Edge will return all of the possible Search Appearance codes for the site for the time period you selected. For example, in the screenshot below, there were impressions and/or clicks for AMP (article and non-rich results), Q&A, How-To, and others for the property I selected.
Copy the codes from the searchAppearance column to a text file so you can reference them in phase two of our tutorial. You will need these codes to export all data by that specific search feature.

Phase Two:

Now we are going to use the searchAppearance codes to export data in bulk for a specific search feature. Click Google Search again, and then Search Analytics. Choose an account and a property again. When you get to the options screen, select the dimensions you want to export (in the Fields tab). For this example, let’s select query (to see the queries yielding How-to snippets in the search results).
Next, go to the Filters tab and find the Appearance field. In that field, enter TPF_HOWTO, which is the code for How-to snippets. If you want to export data for another search feature, just use that code instead.
Next, select the dates you want to run the report for. For this example, I’ll select “Last 28 days”.
Then under Sort/Count, select clicks and then descending in the “sort by” dropdown. This will sort the table by the queries with the most clicks over the past 28 days (that yield How-to snippets).
Then click “Finish”.
Analytics Edge will run and export all of the queries yielding How-to snippets. This can take a bit of time (from a few seconds to a minute or longer) depending on how large the site is and how much data needs to be exported. Note, just a sample of data will be presented in memory in the worksheet (and highlighted in green). You need to “write to worksheet” to show all of the data.
To do that click File and then “Write Worksheet”. Name the worksheet and click OK. You will now see a new worksheet containing all of your data. For this example, I see 26K+ queries that have yielded How-to snippets over the past 28 days. Yep, over 26K!
Congratulations! You just exported search feature data from GSC and blew by the 1K row limit!

Tracking, Using, and Learning From The Data:
Once you export your data by Search Appearance in bulk, you will have full access to all of the queries yielding Q&A, How-To, FAQ snippets, AMP, and more. You can track their position, double check the SERPs to understand the SERP treatment for each feature, understand the click-through rate for each snippet, and more.

For example, you might find that “list treatment” for How-tos is yielding a higher click through rate than the carousel treatment. Or you might find that a certain How-to has a featured snippet in addition to the How-to snippet (a How-to/featured snippet combo). And then you can check metrics based on that situation. You get the picture!

You can see an example of a How-to/featured snippet combo below:

My recommendation is to export data by each search feature for the past 28 days and start digging into the data. Then regularly export the data (like weekly) to understand the changes over time. E.g. changes in metrics, SERP treatment, and more.

Summary – Free Yourself From the 1K Row Limit In GSC By Exporting SERP Feature Data Via Analytics Edge
Now that more and more features are hitting the SERPs, using a tool like Analytics Edge can help you export all of your data, versus just one thousand rows per report. And when you export all of the queries and landing pages per SERP feature, you can glean more insights from the data. If you are using AMP, How-to, Q&A, or FAQs, then I highly recommend exporting your data via a tool like Analytics Edge. I think you’ll dig it.

GG

Exploring Google’s New How-to Snippets In Search And On Smart Displays: SERP Treatment, Fresh GSC Data, Video Templates, Monetization, and more

May 14, 2019 By Glenn Gabe Leave a Comment

Google is always working to enhance features in the search results (SERPs). Over the years, we have seen the SERPs change from ten blue links to a mix of images, videos, featured snippets, one-box results, immersive mobile results, and more. There are many features beyond ten blue links now, which was covered by John Mueller at Google I/O last week.

Some of those new features are minor additions, while others rock the SERP. Well, there’s one new feature that was just released at Google I/O, and it’s a multifaceted feature that gives us a glimpse into the future of Google. It’s called How-to snippets and I’ll cover more about this new feature below.

How-To Snippets: Part of the answer carousel trifecta
Last year at the Google Dance in Singapore, Google announced three new formats for what they call “best answer carousels”. The three features were Q&A, How-to, and FAQ. They are powered by structured data and help users receive rich answers to their questions.

Q&A rolled out already, while How-to has been in testing for several months. Both How-to and FAQ were released last week at I/O, but my focus for this post is on How-to snippets. I believe they are an extremely important feature that impact the search results, and beyond. More about that soon.

There are several sites I work with that have implemented How-to markup (including thousands of How-tos in total) and I’ve been digging deeper to learn more about the new search feature, how it’s being implemented, what it looks like in the SERPs and on Google Assistant smart displays, and how it will impact publishers. Again, it’s live in the SERPs today, so you will definitely see more of How-to snippets as time goes on.

How-to snippets – Providing step-by-step instructions directly in the SERPs and on Google Assistant Smart Displays
First, if you’re on the “featured snippets are taking traffic away from publishers” side of the SEO debate, then you might want to brace yourself. How-to snippets can take up a lot of real-estate and provide a number of steps along the way (directly in the SERPs and on smart displays). As of today, you can see How-tos in the search results, but they have not been rolled out to smart displays yet. That’s coming later this month.

In the SERPs, Google isn’t providing all of the steps from the How-to, but the SERP feature does take up significant real-estate (with visuals when they are provided).

That’s fine for now, but on Google Assistant smart displays, you can walk through all of the steps in the How-to without visiting the site in question. And if you’re monetizing via eyeballs on your own site, then your revenue-warning antenna should have just gone up… :)

Remember, Assistant is extremely important to Google, including its booming smart displays like Google Home Hub, which was just renamed to Google Nest Hub.

Also, Google could expand the number of steps in the SERPs at any point for How-to snippets. So instead of just providing some of the steps like it does now, it could potentially include more (or all) of them. And if that happens, it could throw a major wrench into monetization, email signups, or any other conversion or micro-conversion that would occur on a site.

SERP Treatment: Show me the snippets!
From a SERP treatment standpoint, there are two core treatments as of today. You can see a list version that uses an accordion to reveal information for each step. And below the steps in the list version, you will see one link to visit the site for all of the steps.

Note, these are not all clients of mine… these are just How-to snippets I have seen in the wild from sites that have implemented How-to markup.

And the other treatment is when you have photos for each step. When you do, Google can provide a carousel of steps with visuals. Users can tap each element in the carousel to be taken to that step on the publisher’s website. This is accomplished via the How-to markup added to your pages.

Here are some additional examples of what How-to snippets look like in the wild (via sites that have implemented How-to markup):

As you can see, How-to snippets have eye-catching SERP treatment, provide a number of steps involved to complete a task, and could yield visit-less users for sites with the snippets.

For example, imagine there are just a few steps involved with the How-to. Then all of the steps could be listed right in the SERP. Time will tell how much site owners approve of this new feature… and I’m sure their monetization teams will be watching very closely.

Note, site owners could always just remove the markup if they feel they are losing traffic, but then that leaves important SERP real estate for other sites (like competitors). It’s probably going to be a tough decision for many site owners, especially when Google Assistant smart displays begin showing the full How-to.

On that note, here is what How-tos will look like once they arrive on smart displays. The full How-to will be available and users will be able to view the necessary tools, equipment, products, etc., along with each step in the How-to.

Here’s the video from Google I/O (at 7:40 in the video):

How-to Schema and The Rich Results Test
If you have How-to content on your site and you would like How-to snippets, then your first stop should be the Google Search Developers documentation for adding How-to markup. Google provides a wealth of data and code examples for setting up the necessary code.

For testing your markup, you can use the Rich Results Test, which enables you to either test a url or test a code snippet. Once you do, you can preview the How-to in the search results directly from the tool. It’s a great way to see your markup in action:

You can also view a preview of both types of How-to snippets, including thumbnail view (the carousel) and list view:

No Site? No Problem! Introducing How-To Video Templates – Repurposing Your How-to YouTube Videos For Search and Smart Displays
Some people have joked that in the future, Google might just pull information from publishers and provide that in the SERPs or on smart displays (no website needed). Well, Google sort of announced that at I/O.

If you have How-to YouTube videos and don’t have a site, then you can use video templates provided by Google to add your How-to information in a structured way. Once uploaded, Google can transform your YouTube video into a How-to that can be displayed in the SERPs or on Google Assistant smart displays. You can find more information about this in the developer document that was published when I/O started.

Future Monetization Opportunities:
As mentioned earlier, the full How-to will be available on smart displays, and it’s not hard to believe that there will be an opportunity for monetization via ads, or links to purchase the necessary materials, products, ingredients, etc. Or maybe even sponsored How-tos in the SERPs as a separate ad unit? Anything is possible…

That’s just my opinion, since Google hasn’t announced any monetization opportunities yet. But let’s face it, Google is an ad-driven company and providing advertising in How-tos makes a lot of sense.

Here are two mockups of how Google could provide ads in How-tos (in the SERPS). Smart displays would be clearly different, and could yield ads in between steps, the opportunity to buy materials, and more:

Side Note: How-To + Featured Snippets = MASSIVE SERP Coverage
If you’re wondering about how featured snippets impact How-tos, yes, you can land both featured snippets and How-to snippets in the same SERP. That’s at least for now… And when you do, you can take over massive SERP coverage. You’ll have the initial featured snippet in position 0 and then a How-to snippet that can rank in the top 10. Just an important side note.

Early How-to Data: GSC Performance Reporting
With the launch of How-to snippets, Google has added a new enhancements report for How-to markup and a search appearance filter in the performance reporting for How-to as well. Using both, you can troubleshoot How-to markup on your own pages, while also seeing traffic data like impressions, clicks, CTR, and position for your How-to snippets.

For example, here is a sample enhancements report listing errors, warnings, and valid How-to markup on a site. This image was from Google’s blog post about How-to snippets:

And here is the performance report with How-to listed as a search appearance filter:

I’m only seeing a few days of data so far, but as you can see, there are plenty of clicks to the site from How-to snippets. The click-through rates are pretty strong across some queries, but also pretty average for others. It really depends on the specific SERP, where the How-to snippet ranks, and which other features are displayed alongside the How-to.

For example, you might be competing with featured snippets, video carousels, interesting finds, and other features that can overshadow How-tos. I’ve even seen local one-boxes or local packs overshadow some How-tos.

Also, urls that rank in both featured snippets and How-tos in the same SERP could have skewed metrics. I believe you will see data for both combined in GSC (which can yield higher CTRs and clicks than if the data was just presented for the How-to snippet). I have to analyze more results to see if that’s the case, but I think I’m seeing that already.

So you’ll definitely want to analyze your own SERPs heavily to determine how much impact those How-tos are having (at least in the SERPs). Smart displays are another story. Here are some stats for two different How-to queries. The first ranks higher than the second, which is impacting CTR. They also have different SERP features competing with them.

And here’s an interesting example with an INSANE click through rate of 73%. This is what can happen when you mix breaking news with a How-to (if that ever fits with your site). It’s still early obviously, but that’s a crazy-high CTR!

Again, How-to has not rolled out to Smart Displays yet, and I’m not even sure how that will be reported in GSC (if at all). There should be additional reporting in my opinion, since it’s a different user experience. Most users will just go through the How-to on the smart display and never reach the site, so there may not be traffic data to report. I asked Google’s John Mueller about this in the last webmaster hangout video and he said he wasn’t sure how that will be reported in GSC. It’s still very new and didn’t know how that will be reported yet.

Here is the video (at 28:22 in the video):

I’m sure publishers would like to know how many times a How-to was triggered via Assistant, how far a person made it through the How-to, and if they did end up visiting the site via the link in the Google Home app (which gets added when a How-to is presented). I’ll update this post after How-tos roll out to smart displays.

That’s a good segue to an important topic – feeding Assistant.

The Big Picture – Feeding Google Assistant
Let’s expand our view of what’s going on and explore why would Google want to provide a list of steps like this directly in the search results. We know Google has already heard from many people that featured snippets could be stealing traffic. So why push the limits here with How-to snippets?

Well, Google Assistant is booming and it’s an important part of Google’s future. The family of Google Home devices has been a big success so far, and many people have at least one in their homes. Actually, many have multiple devices between Home, Home-mini, and Home Hub. And Google hasn’t figured out monetization… yet. They absolutely have to and I’m sure there are many people working on that right now.

Even though assistants like Home and Echo have soared, research shows that most people use them for very basic things. For example, to check the weather, get the news, check a sports score, get a quick answer, etc. There are over 80K skills available now for Alexa and over 4K for Google Assistant, but most people don’t even know those skills exist, how to trigger them, etc. So, all of these devices are still used for rudimentary tasks. Again, for now.

Well, one great way to enhance usage, and possibly open up monetization opportunities, is to provide richer answers in a step-by-step guide. For example, if you are looking for help with fixing your mobile phone, you can view and hear a breakdown of the steps involved, view products and materials that could help you fix the phone (with links to buy them), maybe see some ads along the way, etc. And as mentioned earlier, there are many YouTube How-to videos that could be repurposed into How-tos for smart displays. That’s why Google created the How-To Video Templates for video publishers (mentioned earlier).

Therefore, How-to snippets could feed Google Assistant with a ton of step-by-step guides. And by the way, when Google provides special SERP treatment like featured snippets, review snippets, authorship markup of the past, AMP, How-to snippets, and more, publishers typically clear their schedules to implement the necessary technical changes to receive those features. And I’m sure that’s exactly what’s going to happen here with How-to markup. That means Google Assistant could soon be fed millions of How-tos in a structured format (including videos if publishers use the new How-To Video Templates I covered earlier).

It should be very, very interesting to watch how this goes for publishers implementing How-to markup. Will they be ok with feeding the Google beast without much to gain, or will they rip the markup out in order to gain more traffic to their own sites? I’ll be watching closely, that’s for sure.

Summary – Keep an eye on How-to snippets. You can’t miss them in the SERPs (and soon on smart displays)
How-to snippets in the SERPs could be a big deal for Google, publishers, and users. One thing is for sure, it should be interesting to see how this plays out. For those sites that provide How-to content, you should absolutely have this on your radar and watch the rollout very closely. Your traffic could be on the line. And you should test How-to markup on your own site (including videos).

I recommend digging into How-to schema and getting familiar with how it works, how it should be structured, etc. Remember, this is a bigger and badder sibling to featured snippets, with a smart display spin. It will be hard to miss them in the SERPs and via Assistant. I’ll update this post with more information as the rollout continues and I have more data.

GG

A Holly Jolly Load Balancer Christmas – How Google Treated A Major Site Performance Problem From A Crawling And Ranking Perspective [SEO Case Study]

January 23, 2019 By Glenn Gabe Leave a Comment

Ah, the holidays. Who doesn’t love a festive atmosphere, ugly Christmas sweaters, eggnog, singing Christmas carols, and major performance problems that can take down your site?

Wait, what was that last part??

The holidays are always interesting for me SEO-wise. Every few years, I receive an SOS from a site owner right around Christmas Day based on a weird drop in traffic or rankings from Google. This year, it was on December 27 from a company I helped several years ago. It’s a large-scale site that has seen a lot of volatility historically, but has done a good job at turning things around. They have battled major algorithm updates in the past, and now battle new SERP features from Google itself.

Just two days after Christmas, I received a message from the site owner asking if there was some type of update over the holidays. Traffic dipped sharply on Christmas Day, which it typically wouldn’t do based on their niche, and had gone down more and more since. Traffic typically only goes down by about 20% on Christmas Day and it was down 50% this year. That’s typically not the case from a historical perspective for the site and they were worried that an algorithm update impacted the site again.

So I rushed to check their stats, noticed the drop in traffic, and quickly started checking the sources of traffic that dropped. The first thing I noticed was that the drop wasn’t just from Google… it was from all sources. That’s actually a good thing and we could quickly rule out some type of Google algorithm update.

Then I started checking the top landing pages that dropped and it wasn’t long before I noticed a serious performance problem. Some of the pages simply wouldn’t load. Then I searched Google for queries leading to the pages and tried to click through from the SERPs. I experienced the same issue… the site was simply hourglassing.

I messaged the site owner and explained the major performance problems I was experiencing and that the drop was from all traffic sources, and not just from Google. He took that information to his CTO so they could dig in and start isolating the problem. It wasn’t long before I heard back.

A Holly Jolly Load-Balancer Christmas!
No, that’s not the name of the next Christmas Vacation movie, although it does have a nice ring to it. :) The site owner and CTO uncovered a load balancer issue that was causing the performance problems we were seeing on the site. And that was causing a big drop in traffic as visitors couldn’t load many of the pages. And again, this is a site that people visit during the holidays, and on Christmas day, based on the type of content the site provides.

So, the good news is that it wasn’t some type of evil elf algorithm update. But the bad news was that the load balancer issue was causing many pages not to load for users and Googlebot. My client went to work on resolving the load balancer issue, but they couldn’t help but wonder how Google would treat the site from a rankings perspective.

For example, when Google sees a major performance problem over time, and it knows users cannot load the pages, some important questions come to mind. Will Google drop the site’s rankings, will the site plummet in the short-term, only to come back when the performance problems are corrected, will there be long-term damage SEO-wise, and how will crawling and indexing be impacted?

We’ve heard Google’s John Mueller explain situations related to this in the past. For example, John explained that Google can slow crawling when it comes across many 5XX errors. Here’s a tweet from John about this:

And that goes for 503s as well (a response code sites can return when maintenance is being completed and the site is down). John has explained that if Google sees 503s for an extended period of time, it can start dropping urls from the index. Note, returning 503s for a short period of time is totally fine (and is a good approach to use when you know your site will be down for maintenance). Just make sure you’re not returning 503s for an extended period of time.

Since this was a load balancer issue, and some visitors were able to load pages correctly, my client didn’t take the site offline (and didn’t use 503s). But it’s important to understand how Google would treat a site being down over an extended period of time. Here’s a video where John explains this (at 37:28 in the video):

And here’s John explaining how Google can automatically slow crawling when it sees that urls are temporarily unavailable. He also explains that Google will stop crawling if it sees that happening for a robots.txt file. Then Google will automatically increase crawling when the urls return 200s again. Here’s the video of John explaining this (at 38:43 in the video):

So what would happen in this case? Needless to say, we were eager to find out.

Tracking The SEO Impact Of Major Performance Problems
As you can probably guess, I was ultra-interested in seeing how this played out SEO-wise. So, I began tracking the situation on several levels to see how the load balancer issue was impacting the site from an SEO standpoint. For example, would rankings drop, would crawling slow down, and how would this all look when the performance problems were fixed?

Below, I’ll take you through the timeline, with data and screenshots, to show you how Google treated the problem. Note, every site is different, so this may not apply to all performance problems you run into. But, it should help you understand how those performance problems could impact crawling, indexing, and ranking. It’s also worth noting that the duration of performance problems would surely impact this situation. For this case, the site owner and CTO completely fixed the problem within seven days. But, it wasn’t all fixed at one time. More on that soon. Last, I unfortunately don’t have logs from the site yet. I’m still working on getting them, though, and will update this post if I do.

Blue Christmas – The Timeline of Events (with data and screenshots)
First, here is the drop in traffic when comparing year over year. The site usually doesn’t drop very much during Christmas Day (or week). You can see a seasonal drop of about 20% when compared to normal weeks, where this year yielded a drop of 50%. Yep, something was up…

YOY trending shows the unusual drop on Christmas Day in 2018.

Crawl Stats in GSC:
Here was the spike in time downloading a page and the drop in pages crawled per day. Pretty amazing to see that graph, right?

Crawl stats in GSC show a spike in time spent downloading a page
while pages crawled per day drop off a cliff.

And here were the impressions and clicks in GSC (which remained strong). This supports the notion that rankings were not being impacted during the performance problem. Impressions would have dropped if that were the case. I was also checking rankings manually and saw rankings remain strong overall. It seems that Google was pretty cool with understanding there was a performance problem and didn’t quickly impact rankings (even when it knew some users were running into page loading problems).

GSC Performance reporting reveals stable impressions and clicks from Google
while the performance problem was in place.

And here was search visibility trending in SEMrush over the timeframe, which supports the fact that rankings were not impacted in the short-term. Search visibility was stable throughout the performance problem:

Search visibility trending in SEMrush is stable throughout the problem (no drop in rankings).

And here is Sistrix trending, which is also stable throughout the problem:

Search visibility trending in Sistrix is also stable throughout the problem.

Note, the load balancer problem was not fixed all at one time. The situation improved over a five to seven-day period. So here are the crawl stats as the issue started to get resolved. Notice crawling increases, but not back to normal. That’s because the performance problems were still there to some extent. Notice time downloading a page improves, but also not back to normal.

Pages crawled per day increase as time spent downloading a page decreases
(as the load balancer problem is being fixed).

Pages crawled per day and time spent downloading a page continue
to improve as the performance problem is fixed.

And then once the problem was completely fixed, notice that the crawl stats return to normal. Time downloading a page drops back down to normal levels and pages crawled per day rise to normal levels. Awesome, all is good again in Google Land.

Pages crawled per day and time spent downloading a page return to normal levels
when the load balancer problem was completely fixed.

And here is what Google organic traffic looked like from before the load balancer issue to after it was resolved. Notice the drop during Christmas, a further drop, and then traffic returns to normal after the problem was fixed. There was no impact to Google organic traffic, rankings, etc. based on the performance problem. That was great to see.

Traffic from Google organic returns to normal once the
load balancer problem was completely fixed.

Key learnings… and Spiked Eggnog:
Needless to say, it was fascinating to watch how Google reacted to the major performance problems over time. Here are some key learnings from the situation:

Google understands that temporary glitches can happen (like the performance problems this site was experiencing).
Google can slow crawling, when needed, and check back to see when it can crawl more. The crawl stats in GSC clearly showed that happening.
For this situation (only a week or so), the site wasn’t impacted SEO-wise. Rankings remained strong, search visibility was stable, and as the problem was fixed, Google returned crawling back to normal levels.
Google organic traffic then returned to normal levels as the performance problem was fixed completely. There was no impact rankings-wise based on this incident.
That said, if this problem remained beyond a week or two, it’s hard to say how rankings would be impacted. Google could very well begin to drop rankings as it wouldn’t want to send users to a site or pages that don’t resolve. But, if you have a short-term performance issue, it seems you should be ok. I would just work to fix the problems as quickly as you can.
This is a reminder that Murphy’s Law for SEO is real. The load balancer problem happened on Christmas Day. You just need to be prepared for Murphy to pay a visit. And then move quickly to rectify the problems he brings along.

Summary – All I want for Christmas is to never deal with a load balancer issue again.
If you run into serious performance problems and are worried about the SEO-impact, know that Google does understand that bad things happen from time to time. As Google’s John Mueller explained, Google can slow crawling while the problems exist and then check back to see when it can return crawling to normal.

For this specific case, that’s exactly what we saw. Rankings and search visibility remained strong, but crawling slowed. Then as the performance problems were fixed, crawling returned back to normal. And all was good again in Google Land.

So, until next holiday season… here’s ho-ho-hoping you don’t run into this type of problem anytime soon. I know everyone involved with this case feels the same way. :)

GG