The Internet Marketing Driver

The September 2019 Google Core Update – Case studies, health and medical volatility, Discover impact, “baby algorithms”, and revisiting the “kitchen sink” approach to remediation

By Leave a Comment

Google typically pushes broad core ranking updates only a few times per year, and on September 24, 2019, they pushed yet another. Google’s Danny Sullivan pre-announced the update on Twitter and explained like previous core updates, it would be a global update and could take several days to fully roll out.

Although it was quiet initially, we began to see impact a full day or two into the rollout, with some sites surging and others dropping across categories and countries. That shouldn’t shock anyone who follows core updates closely… I often explain that they can make the earth shake.

In this post, I’ll explain some interesting case studies I’ve worked on, including sites that increased and other sites that dropped. These are sites that I’ve helped after they reached out to me about negative impact during core updates, so I have a lot of knowledge about the sites overall, their history during previous core updates, the types of problems each site had across categories, how they chose to tackle those problems, the speed at which those changes rolled out, and more.

Beyond the cases of surges and drops, I’ll also cover the volatile health and medical niche (including alternative health), the impact to the ever-growing Discover feed, how technical problems could cause quality problems, and I’ll underscore the importance of employing a “kitchen sink” approach to remediation (where it’s important to surface ALL potential problems and fix them as quickly as you can).

I’ll explain why that makes a lot of sense given how Google’s core ranking algorithm works, which was reinforced by Google’s Gary Illyes at Pubcon last Thursday (when he brought up millions of “baby algorithms” working together).

Definitely grab a cup of coffee, or two, we’ve got a lot to cover.

Google’s Core Ranking Algorithm – Millions of “baby algorithms”
Google’s core ranking algorithm is an interesting animal. Google has always explained that it’s taking many factors into account when evaluating sites. Gary Illyes just explained at Pubcon that Google’s core ranking algorithm is made up of millions of “baby algorithms” that work together to spit out a score. That makes complete sense given what I’ve seen in the field when helping companies that have been impacted.

I’ve often said there’s never one smoking gun… Instead, there’s usually a number of things that can be improved across a site, from content quality to user experience to aggressive and disruptive ads to technical SEO problems causing quality problems, and more. And beyond that, Google can tinker with many algorithms, which can obviously cause some sites to surge or drop without any changes being implemented.

Yep, welcome to the wonderful world of core updates.

My advice has always been to thoroughly analyze your site through the lens of quality, which can mean several things. Don’t just cherry pick a few things to improve. Instead, improve the site overall.

That’s also why running a user study is a great idea, and can be incredibly powerful. You can receive objective feedback about your site content, user experience, advertising situation, learn how much people trust your site, understand if they believe you are an expert in your niche, and more. You can read my post about running user studies through the lens of core updates to review some of the feedback we received for a health/medical client. Some of that feedback was eye-opening for my client.

Google’s Algorithms Have Become More Critical Of Health And Medical Sites (and the adjustments keep coming…)
I’ve covered the health and medical niche a lot in my previous posts about core updates (especially since the Medic update in August of 2018). Since Medic, the health and medical niche has seen extreme volatility. There were many sites in that category seeing massive fluctuations in rankings. Some were dropping by over 80% while others surged like crazy.

For example, check out the crazy swings in search visibility below for several sites in the health/medical niche:

Health and medical site dropping and surging during core updates.
Another health and medical site surging and dropping during core updates.
Yet another example of crazy volatility with a health and medical site during core updates.

With each subsequent core update, many sites in the health and medical niche experienced major turbulence, with some sites surging with one update, only to drop with the next. And that’s even for sites that were implementing a lot changes to improve overall.

It wasn’t until May of 2019 that we specifically heard about the niche from Google’s John Mueller. He explained that Google’s algorithms have gotten more critical for health and medical sites over the past year or so. And, Google has gotten much better at determining quality in the health and medical niche. I’m really glad John explained that, since again, we had seen crazy volatility in the category leading up to that statement.

Here is John’s comment from the webmaster video hangout:
“And then of course there are other kinds of websites where I’d say maybe our algorithms in the last year or so have been focusing more on and where they’re a bit more critical. For example, one area that I’ve seen mentioned a lot is the whole medical space. Where I see people posting online talking about authority, expertise, … those kinds of things where I’d say that definitely makes sense. Where in the past it was really hard for us to judge kind of the quality of the medical or medically-oriented site. And over time our algorithms have gotten better in that regard.”

I’ve been extremely vocal since the June core update that we would see a lot of volatility with the next core update in health and medical (as Google adjusts its algorithms again based on previous impact). That’s because there were some sites that dropped heavily in June that probably shouldn’t have (in my opinion). And then we had many alternative health sites that had gotten hammered by previous core updates (losing 80%+ of their search visibility) that also seemed to get hit a little too hard. So, I had a feeling that Google would be making adjustments with the September core update.

And as I thought, there was as ton of volatility in the health and medical space. A number of major health sites were impacted heavily, while a number of alternative health sites rebounded (but didn’t increase much compared to where they were prior to getting hit). So they increased, but that increase pales in comparison to where they once were visibility-wise.

Here are some examples of the movement in the health and medical space during the September 2019 core update:

Controlling what they can control…
I have been helping a number of companies in the health and medical space over the past year and many of those companies have been working hard on improving their sites overall. That includes improving content, making sure that content is written by medical professionals, making sure they have a strong medical review board, cutting down on ad aggressiveness, improving the user experience, earning powerful links (naturally), and more.

But, health and medical sites can only control what they can control. They can’t control Google’s core ranking algorithm (which again, it is composed of many different algorithms, some of which can end up impacting health and medical sites significantly). In other words, those sites can absolutely see volatility based on Google tinkering with its algos alone… That doesn’t mean health and medical sites shouldn’t be looking to improve overall. They definitely should. But it’s an important point that many site owners need to understand.

Control what you can control and hope that improving the site overall can yield stronger scores as the millions of algos that make up Google’s core ranking algorithm do their thing.

So, always look to significantly improve your site and keep those changes in place over the long-term. But also understand that some of this is out of your control. That’s exactly what we’ve seen with health and medical and I’ll cover some cases below.

Interesting cases from the September 2019 core update:

Reviews website: Technical problems causing content quality problems
The first site I’ll cover is a large-scale review site that was negatively impacted by a previous core update and had struggled to regain visibility. As with any large-scale site, you never know what you’re going to find until you truly dig in.

It wasn’t long before I uncovered a big technical problem that was causing tens of thousands of lower-quality pages to be indexed. This finding was actually part of my first deliverable for the client. It was a large-scale canonical problem that underscores the point that rel canonical is just a hint. Google can, and will, make its own decisions. And those decisions could end up adding many lower-quality or thin pages to the index. The problem impacted tens of thousands of pages across the site.

And remember, Google is on record explaining that every page indexed is taken into account when evaluating quality. You can watch John Mueller explain that at 10:06 in this video or at 25:20 in this video. That’s why it’s important to focus on “quality indexing”, or making sure your highest-quality content is indexed while making sure low-quality or thin content isn’t indexed.

Canonical problems causing quality problems. Google indexing many low-quality pages that were being canonicalized to other urls (but it was ignoring rel canonical).

So that was a big issue for the site, but that was just one issue. There were many other problems that could be potentially impacting the site from a core update standpoint. For example, there were render issues on many pages across the site (where some content wasn’t being rendered and indexed properly), important areas of the site were being blocked from crawling, there was a low-quality/thin content problem across certain areas of the site, and a host of other issues that could be causing quality issues.

The company employed a “kitchen sink” approach and began tackling many of the issues that were surfaced. And during the September update, they saw a nice increase in rankings and traffic. As of today, the site is up 31% since the update rolled out.

Here is their increase in search visibility:

Surge in search visibility with the September core update.

Here is the increase in GSC:

Increase in impressions and clicks with the September 2019 core update.

The most intriguing part about this case is the massive canonical situation that led to many low-quality pages getting indexed. That accounted for tens of thousands of pages. I’ve always said that technical problems could cause quality problems, and this was a great example of that happening.

News publisher surges after dropping heavily in June & “Discovers” an interesting side-effect:
The next site I’ll cover is a news publisher that was heavily impacted by the June core update. The site dropped by 74% overnight in June and the site owner reached out to me for advice. I wasn’t able to take them on as a client due to my schedule, but I did take a look at the site and provided several recommendations.

News publisher drops by 74% after June core update.

As with many sites that are impacted by core updates, there were a number of things that could be problematic (and not just one). For example, there was an aggressive ad situation, including a potential above the fold (ATF) problem. There were many ads on each page, and some larger ones at the top of articles pushing the main content down the page. In addition, there was a lot of older thin articles on the site. And some were written by writers without a lot of expertise.

So, the site owner went through the process of analyzing and removing a lot of thin content while continuing to publish higher quality content. I see a drop of about 2K urls in the coverage report when reviewing valid and indexed pages (showing the drop of thinner articles).

Beyond that, the site is in a dynamic niche and has been earning links from other sites in that niche (naturally, based on the content). That can also help from an authoritativeness and trust standpoint. Remember, Google’s Gary Illyes explained in the past that E-A-T is heavily influenced by links and mentions from well-known sites. Also, Google published a whitepaper where it explained more about how it assesses expertise, authoritativeness, and trust. Google explained that one of the best-known signals it uses is PageRank, or links across the web, to understand authoritativeness. So, the right links do matter.

The site surged during the September update, including both Search and Discover traffic. Sessions from Google organic search increased from 600 visits per day to over 6K per day (and even topped 10K per day based on certain stories being run).

By the way, Google has explained that Discover visibility can be impacted during core updates and this is a great example of that happening. Discover traffic rose from the dead… Here is Google Analytics trending, and both the Search Performance reporting and Discover reporting from GSC:

News publisher surging from 500 sessions per day to over 10K on some days.
Clicks and impressions surge in Google Search (Discover screenshot is below).
Discover visibility rose from the dead with the September core update.

And if you want to see what MAJOR Discover impact looks like based on core updates, Chris Hawley shared a screenshot of a drop from 100K clicks per day to 0 from Discover after the September core update. I don’t know if that has changed for them, but that’s an insane drop:

Alternative Health Site Increases, But Pales In Comparison To Previous Levels:
The next site I’ll cover is an alternative health site that got obliterated during the March 2019 update, losing over 55% of its traffic overnight. I covered the health/medical situation earlier, and the alternative health niche within that category is a crazy-volatile space.

Alternative health site drops heavily during March 2019 core update.

I explained earlier that Google could tinker with the millions of “baby algorithms” that make up its core ranking algorithm, and that alone, could cause a lot of volatility. And when mixed with the right site changes, certain sites can see a nice increase after an update. I believe we saw that again with the September 2019 core update for certain sites.

That being said, with any site in the health/medical space, it’s hard to say how much impact is coming from improving a site overall versus Google turning the dials on its algos. This case I’m covering is representative of many others out there (especially in alternative health).

While auditing the website, I surfaced many issues during the audit that could be impacting the site from a quality standpoint. There was thin and low-quality content across the site, technical problems, aggressive advertising problems, user experience issues, performance problems, and more.

The site owners were somewhat handcuffed and could not make as many of the changes as they wanted to, but they did implement a number of changes across the site.

Regarding the changes they didn’t implement, I do understand why they held their ground in some areas. In a nutshell, they aren’t going to sell their souls to rank better in Google. They believe in something, and are sticking to their guns. The downside, of course, is a trickle of Google traffic compared to where they were in the past. Again, alternative health is a tough niche to deal with.

With the September 2019 core update, the site was positively impacted and increased by 33%. That’s great, but is still pales in comparison to where they once were. Remember, they dropped by over 55% during the March core update. When the surge began on September 25, I was really hoping to see the site recover more… so the increase was pretty good, but not great.

Alternative health site increases during the September 2019 core update.

A note about scientific consensus and alternative health sites:
There has been a lot of talk about whether Google is using scientific consensus when evaluating and ranking sites in health and medical. Again, with core updates since August of 2018 (Medic), many sites that fall into “alternative health” have gotten hammered. And many of those sites produce content that goes against scientific consensus.

Google Gary Illyes was just asked about this during the Q&A with Jennifer Slegg at Pubcon. His answer was interesting. He first said that it’s an area he doesn’t want to provide too much information. That’s a telling statement by the way. :) Next, he said that some of Google’s algorithms use entities. And entities can map to terms.

Here are some tweets from the session covering Gary’s answer:

And then Lily Ray tweeted an interesting comment from Bill Slawski (from his presentation at Pubcon). It made a lot of sense to think about the situation this way:

Again, alternative health is one of the toughest areas for SEO right now, with Google better-determining quality algorithmically for health/medical sites (over the past year or so since the Medic update) and possibly using signals to understand scientific consensus. Nobody is sure how they are doing this… but the volatility in the health/medical niche has been insane since the Medic update.

News publisher – Maintaining strong SEO hygiene as algorithmic battle scars remain
The next case is a site that got hammered by medieval Panda in the past, but has worked hard ever since to make sure the site is the best it can be. It’s a large-scale news publisher with millions of urls indexed and hundreds of urls added each day.

After getting smoked by Panda in 2014 and losing 65% of its traffic overnight, I have worked with the company to continually audit and improve the site. It’s a complex and large-scale site, so there are many gremlins that can rear their ugly heads (at any time). But since we are continually analyzing the site, we tend to pick up those problems relatively quickly and rectify them. That includes content quality problems, technical SEO issues, and more.

The site surged back in 2014 after making a ton of changes to improve and has done extremely well since then. Then during the June update, it surged once again, and now with the September update, it surged even more.

Anything can happen in the future, but I believe maintaining strong SEO hygiene has helped this site drive over 210K sessions per day from Google organic, while seeing surges during the previous two core updates. It’s great to see.

The first screenshot below is from the June core update and the second is from the September core update:

News publisher increasing with the June core update,
and then increasing more in September (see screenshot below).
News publisher increases more during the September 2019 core update.

Health and Medical Site – major negative impact during multiple core updates
The next case involves a health and medical site that has gotten hammered by multiple core updates in a row. There were many problems I surfaced during the audit, including content quality issues, aggressive advertising problems, copied content, major performance issues, UX issues, lacking E-A-T, and more. There were a lot of changes that needed to be implemented.

But time was not on their side… The site had barely implemented any changes when the September update rolled out, which is a dangerous situation for a site that Google clearly had issues with during previous core updates.

During the September core update, the site actually dropped more. It was already down heavily since the March 2019 core update, but now it dropped another 37%. So the site is down 82% since before the March core update.

Health and medical site dropping heavily during two core updates in 2019.
Health and medical site dropping even more during September core update.

It’s a great example that underscores the importance of implementing significant improvements in quality overall to see positive impact during core updates.

I’m eager to see how the site does after implementing more of the changes that were mapped out (including cutting down the ad aggressiveness in a big way). Remember, Google will need to see those changes in place over the long-term, so I hope they can see positive impact during the next core update.

Also, we might be running a user study soon, which would help the site owners get a strong feel for how objective users view the site (through the lens of core updates). I think that would be greatly beneficial.

Summary – Revisiting the “kitchen sink approach” to remediation
I hope this post was helpful for understanding more about core updates, how sites can be impacted, changes that need to be implemented, and the unique impact to the health and medical niche since August 2018. It’s been fascinating to analyze the updates over the past year, to say the least.

Regarding improvement, I have covered using a “kitchen sink” approach to remediation heavily in my posts about core updates and I still believe that’s the best approach. Don’t look for one or two things you can improve… there’s never one smoking gun. Instead, there’s typically a battery of smoking guns.

Look to significantly improve your site overall. Don’t try to match competitors, try to beat them in every category. If you do, then there’s a greater chance at Google picking up many different signals about your site that could lead to improvements during major core updates.

In addition, I highly recommend running a user study through the lens of core updates. Learning from objective users via audio, video, and written responses could yield many changes that you might miss via other methods (or just help confirm the changes you have planned).

Final note: Make the right changes for your users and keep those in place for the long-term
After implementing changes based on an audit and/or user study, don’t roll back those changes too quickly. Google is on record explaining that content impacted by a core update won’t recover until another core update rolls out. That’s exactly what I’ve seen while helping companies with core updates.

Google about recovering from a core update.

So don’t roll out changes for two weeks and think it’s not working. Implement the right changes for your users and keep them in place for the long-term. That’s how you can see positive impact during subsequent core updates.

If you do this, great things can happen. You just need patience… and then another core update to roll out. :) Good luck.

GG

Google’s Core Algorithm Updates and Copied Content: The Domino Effect of Negative Impact

By Leave a Comment

There’s never a dull moment in Google Land. And that’s especially the case for health and medical sites over the past year starting with the Medic Update in August of 2018. Since then, we have seen several major core updates, which seem to include a new method of evaluating quality for health and medical sites. Google’s John Mueller explained more about the changes in a webmaster hangout video in May.

That algorithmic change has led to some insane volatility for various sites in the health and medical niche. Some are surging and dropping with every update, which is a clear sign that Google is heavily tinkering with those algorithms (turning the dial up and down to try and find the right balance). I’ve been pretty vocal that I believe Google hasn’t figured this out yet… and I believe we’ll see even more volatility in the health space with the next core update (which we are due for).

I’ve had the opportunity to help a number of health and medical sites over the years, and even more since last August when the Medic Update landed. It’s been fascinating to see the impact, surface problems across those sites, watch some sites surge, others drop more, and some ride the “Google roller coaster” (surging and dropping with each core update). Like I said earlier, there’s never a dull moment in Google Land.

A Multi-Core Victim with an interesting content problem:
I recently started helping another health and medical site that has gotten hammered over several core updates. Their traffic dropped 54% during the June core update, and they are down 67% since the March core update. After the latest drop in June, the site owners finally decided to have someone come in and heavily analyze the site through the lens of Google’s core updates to see what’s going on.

Here are the drops from the March and June core updates:

Drop in clicks and impressions during two core updates.

During my first wave of analysis, I surfaced a huge problem that I’m sure is contributing to the site’s drop during core updates. Sure, there’s never one smoking gun with Google’s core updates… there is typically a battery of problems. For example, I’m still heavily analyzing the site and surfacing more issues as I write this post. But, the issue I found was big enough that I actually stopped analyzing the site to craft a separate deliverable just about the topic. That’s what I’m going to cover in my post.

The Find – The Domino Effect of Negative Impact From Copied Content
One of the first things I like to do when helping a company that was impacted by a core update is to run a traffic drop report (what I previously called a Panda Report). The report enables me to see the content that dropped the most after a major core update rolls through. It can often reveal glaring problems across once-popular content on a site.

It wasn’t long before I noticed a disturbing trend. There were many articles that dropped that were copied from third-party health sites and blogs. And to make matters worse (and what should be obvious to SEOs reading this post), the copied pages were fully indexable. In other words, they weren’t being noindexed and they weren’t being canonicalized to the original articles. By the way, it’s not like those tactics make copying content better… but it at least decreases the chance of those pages outranking the original. 

Here is an example of one page copied that was ranking for over 1K queries in the top 10 results BEFORE the March core update pummeled the site (and the copied content):

A copied page ranking for 1K+ queries in the top 10 results
BEFORE the March core update.

Important reminder: YMYL sites held to a higher standard
Remember, this is a site focused on “your money or your life” (YMYL) topics in health and medical. And many sites in the health/medical niche have gotten hammered since the Medic update, seeing a big drop in rankings in traffic during several core updates in 2019. It’s a volatile space, that’s for sure.

Based on that impact, many health and medical sites have been working hard to publish killer content, showcase their expertise, hire medical experts to write and review their content, build medical review boards, and more. So, having content that’s copied from other sites in their niche is clearly not a good thing. And some of those sites are the biggest players in the health and medical category.

In total, there were hundreds of articles that fell into this category. And to clarify, there was original content on the site beyond the copied content. It’s not like the entire site was filled with copied content. But again, there were hundreds of copied articles that were freely indexable (and some were ranking well at certain points in time).

When checking the trending of those copied articles over time, many had dropped heavily in rankings and organic search traffic. Trending for many of those specific pieces of content looked like this:

A copied piece of content dropping during the June core update.

The Domino Effect: Wait, were those sites hit too?
When checking the content that was copied, I noticed some of those articles weren’t high quality. So I decided to check the sites where the content was being copied from. And low and behold, several of those sites had been hit by major core updates as well! Therefore, we had a classic SEO double whammy. The site I was analyzing was copying content from other sites in the niche, but it was also copying content from several sites that were also being hit hard by core updates.

Here is search visibility trending for two of those sites.

A site where content was copied from dropping during major algorithm updates.
Another site where content was copied from dropping during the Medic update.

Copied Content and Major Google Algorithm Updates (Including Medieval Panda)
I’ve seen copied content, and the incorrect use of consuming syndicated content, cause massive problems over the years. I wrote about it (check the Q&A) when medieval Panda roamed the web and I’ve seen it cause problems with major core updates as well. It makes sense. If Google believes you are copying another site’s content, and then ranking for that content, then Google’s quality algorithms can have a big problem with that situation. And in a YMYL world, it can be a very serious issue.

In case you are wondering, Google has explained the problems with copying content a number of times, including publishing a section in its Quality Rater Guidelines (QRG). Google also has a Pirate algorithm for extreme cases, which I have analyzed heavily.

For example, John Mueller has explained that if Google believes a majority of your content is copied, then it can lose trust in your site. In extreme cases, the webspam team might even get involved. Again, I have seen this play out many times over the years (algorithmically and via manual actions). It’s not pretty.

Here is a video of John explaining this (at 26:03 in the video):

And here is a section from the QRG about copied main content:

DMCA threats, but no official takedowns.
After surfacing this situation, I wondered how many DMCA takedowns the site received. It ends up not many were filed for some reason. I found out that there were a few threats of using DMCA takedowns, but none of the sites ever acted on those threats. I believe the site did remove those articles when contacted by third-parties whose content had been copied, but there were no official DMCA takedowns filed.

No DMCA takedowns filed.

By the way, that should have been a sign that something wasn’t right… When you are being threatened with DMCA takedowns, that’s never a good sign.

Moving forward: How to resolve the issue
In order to rectify the situation, there are multiple paths the site can take. I’ll cover them below:

1) Nuke it: 404, 410
First, I’m typically aggressive with core update remediation. So, you can probably guess what my recommendation was. I would nuke the copied content via 404s or 410s. Let’s face it… it’s not your content to begin with! Remember, Google takes every page indexed into account when evaluating quality, so removing low-quality content is always a good thing (including copied content).

2) Seek permission and then canonicalize:
Next, if a site really wants to provide that third-party content for some reason, then the site owners can seek permission to use that content and then use rel canonical pointing back to the original page (basically canonicalizing the urls on your site to the original urls). So, this involves requesting authorization to consume syndicated content and then properly canonicalizing the url to the original.

3) Seek permission and noindex:
You could also seek permission for publishing the content on your site and then noindex that content. Remember, if it’s not indexed, it can’t hurt the site quality-wise. And to be clear, this is AFTER receiving approval for publishing the content from the original author and site. Don’t just go and start copying content from anywhere thinking it’s fine. It’s definitely not fine.

4) Remove and 301 redirect to extremely relevant content (if possible)
You could also 301 redirect the old pages you are removing to extremely relevant content on your site. But keep in mind that Google can simply treat the old pages as soft 404s if you redirect to non-relevant content. You can read my case study about that, and my recent post about Google ignoring rel canonical when the content wasn’t the same or extremely relevant. Those are important points to consider.

Summary: The moral of the (SEO) story:
Don’t copy another site’s content. Don’t try to rank based on content that’s not yours. It’s bad karma and as the saying goes… “karma will take care of it”. In this situation, some of those articles were copied from sites that also got hammered by Google’s core updates. So those pieces of copied content could have contributed to the quality problems that this site is dealing with now. Again, it’s a great example of an SEO double whammy.

Again, I’m definitely not saying this is the only problem the site is dealing with. I’m surfacing a number of issues across categories, but having hundreds of copied articles on the site is definitely not helping the situation.

Instead of taking the easy path and copying content, you should always be striving to create 10X content that blows your audience away, that can naturally attracts links, spark social sharing, and drive searches for your brand plus the topic. If you do that, then you won’t have to worry about the domino effect of negative impact from copied content – or about bad karma. Instead, start building good karma… that’s how you win.

GG

NOT taking the (canonical) hint: How to estimate a low-quality indexing problem by page type using Google Analytics, Search Console, SEMrush, and Advanced Query Operators

By Leave a Comment

During a recent crawl analysis and audit of a large-scale site that was negatively impacted by Google’s core updates, I surfaced an interesting SEO problem. I found many thinner and low-quality pages that were being canonicalized to other stronger pages, but the pages didn’t contain equivalent content. As soon as I saw that, I had a feeling what I would see next… since it’s something I have witnessed a number of times before.

Many of the lower-quality pages that were being canonicalized were actually being indexed. Google was simply ignoring rel canonical since the pages didn’t contain equivalent content. That can absolutely happen and I documented that in a case study a few years ago. And when that happens on a large-scale site, thousands of lower-quality pages can get indexed (without the site owner even knowing that’s happening).

For example, imagine a problematic page type that might account for 50K, 100K, or more pages indexed. And when Google takes every page indexed into account when evaluating quality, you can have a big problem on your hands.

In the screenshot below, you can see that Google was ignoring the user-declared canonical and selecting the inspected url instead. Not good:

Google ignoring rel canonical.

In addition to just getting indexed, those lower-quality pages might even be ranking in the search results for queries and users could be visiting those pages by mistake. Imagine they are thin, lower-quality pages that can’t meet or exceed user expectations. Or maybe they are ranking instead of the pages you intend to rank for those queries. In a case like that, the problematic pages are the ones winning in the SERPs for some reason, which is leading to a poor user experience. That’s a double whammy SEO-wise.

Quickly Estimating The Severity Of The Problem
After uncovering the problem I mentioned above, I wanted to quickly gauge how bad of a situation my client was facing. To do that, I wanted to estimate the number of problematic pages indexed, including how many were ranking and driving traffic from Google. This would help build a case for handling the issue sooner than later.

Unfortunately, the problematic pages weren’t all in one directory so I needed to get creative in order to drill into that data (via filtering, regex, etc.) This can be the case when the urls contain certain parameters or patterns of characters like numerical sequences or some other identifying pattern.

In this post, I’ll cover a process I used for roughly discovering how big of an indexing problem a site had with problematic page types (even when it’s hard to isolate the page type by directory). The process will also reveal how many pages are currently ranking and driving traffic from Google organic. By the end, you’ll have enough data to tell an interesting SEO story, which can help make your case for prioritizing the problem.

A Note About Rel Canonical – IT’S A HINT, NOT A DIRECTIVE
For any site owner that’s mass-canonicalizing lower-quality pages to non-equivalent pages, then this section of my post is extremely important. For example, if you think you have a similar situation to what I mentioned earlier and you’re saying, “we’re fine since we’re handling the lower-quality pages via rel canonical…”, then I’ve got some potentially bad news for you.

As mentioned earlier, rel canonical is just a hint, and not a directive. Google can, and will, ignore rel canonical if the pages don’t contain equivalent content (or extremely similar content). Again, I wrote a case study about that exact situation where Google was simply ignoring rel canonical and indexing many of those pages. Google’s John Mueller has explained this many times as well during webmaster hangout videos and on Twitter.

And if Google is ignoring rel canonical on a large-scale site, then you can absolutely run into a situation where many lower-quality or thin pages are getting indexed. And remember, Google takes all pages indexed into account when evaluating quality on a site. Therefore, don’t just blindly rely on rel canonical. It might not work out well for you.

Walking through an example (based on a real-world situation I just dealt with):
To quickly summarize the situation I surfaced recently, there are tens of thousands of pages being canonicalized to other pages on the site that aren’t equivalent content-wise. Many were being indexed since Google was ignoring rel canonical. Unfortunately, the pages weren’t located in a specific directory, so it was hard to isolate them without getting creative. The urls did contain a pattern, which I’ll cover soon.

My goal was to estimate how many pages were indexed and how many were ranking and driving traffic from Google organic. Remember, just finding pages ranking and driving traffic isn’t enough, since there could be many pages indexed that aren’t ranking in the SERPs. Those are still problematic from an SEO standpoint.

The data would help build a case for prioritizing the situation (so my client could fix the problem sooner than later). It’s a large-scale with many moving parts, so it’s not like you can just take action without making a strong case. Volatility-wise, the site was impacted by a recent core update and there could be thousands (or more) lower-quality or thin pages indexed that shouldn’t be.

With that out of the way, let’s dig in.

Gauging The Situation & The Limits Of GSC
To gauge the situation, it’s important to understand how big of a problem there is currently and then form a plan of attack for properly tackling the issue. In order to do this, we’ll need to rely on several tools and methods. If you have a smaller site, you can get away with just using Google Search Console (GSC) and Google Analytics (GA). But for larger-scale sites, you might need to get creative in order to surface the data. I’ll explain more about why in the following sections.

Index Coverage in GSC – The Diet Coke of indexing data.
The index coverage reporting in GSC is awesome and enables you to view a number of important reports directly from Google. You can view errors, warnings, pages indexed, and then a list of reports covering pages that are being excluded from indexing. You can often find glaring issues in those reports based on Google crawling your site.

Based on what we’re trying to surface, you might think you can go directly to the Valid (and Indexed) report, export all pages indexed, then filter by problematic page type, and that would do the trick. Well, if you have a site with less than 1,000 pages indexed, you’re in luck. You can do just that. But if you have more than 1,000 pages, then you’re out of luck.

GSC’s Index Coverage reporting only provides 1,000 urls per report and there’s no API (yet) for bulk exporting data. Needless to say, this is extremely limiting for large-scale sites. To quote Dr. Evil, it’s like the Diet Coke of indexing data. Just one calorie… not thorough enough.

Search Console API & Analytics Edge
Even though exporting urls from the Valid (and Indexed) category of index coverage isn’t sufficient for larger-scale sites, you can still tap into the Search Console API to bulk export Search Analytics data. That will enable you to export all landing pages from organic search over the past 16 months that have impressions or clicks (basically pages that were ranking and driving traffic from Google). That’s a good start since if a page is ranking in Google, it must be indexed. We still want data about pages indexed that aren’t ranking, but again, it’s a start.  

My favorite tool for bulk exporting data from GSC is Analytics Edge. I’ve written about Analytics Edge multiple times and you should definitely check out those posts to get familiar with the Excel plugin. It’s powerful, quick, reasonably priced, and works like a charm.

For our situation, it would be great to find out how many of those problematic pages are gaining impressions and clicks in Google organic. Since the pages are hard to isolate by directory or site section, we can export all landing pages from GSC and then use Excel to slice and dice the data via filtering. You can also use the Analytics Edge Core Add-in to use regex while you’re in the process of exporting data (all in one shot). More about that soon.

Exporting landing page data from GSC via Analytics Edge

A Note About Regex For Slicing And Dicing The Data
Since the pages aren’t in one directory, using regular expressions (regex) is killer here. Then you can filter using regular expressions that target certain url patterns (like isolating parameters or a sequence of characters). To do this, you can use the Analytics Edge Core Plugin in conjunction with the Search Console connector so you can export the list of urls AND filter by a regular expression all in one macro.

I won’t cover how to do that in this post, since that can be its own post… but I wanted to make sure you understood using regex was possible.

You can also use Data Studio and filter based on regular expressions (if you are exporting GSC data via Data Studio). The core point is that you want to export all pages from GSC that match the problematic page type. That will give you an understanding of any lower-quality pages ranking and driving traffic (that match the page type we are targeting).

Now let’s get more data about landing pages driving traffic from Google organic via Google Analytics.

Google Analytics with Native Regex
In order to find all problematic page types that are driving traffic from Google organic, fire up GA and head to Acquisition, All Traffic, and then Source/Medium. This will list all traffic sources driving traffic to the site in the timeframe you selected. Choose a timeframe that makes sense based on your specific situation. For this example, we’ll select the past six months.

Then click Google/Organic to view all traffic from Google organic search during the timeframe. Now we need to dimension the report by landing page to view all pages receiving traffic from Google organic. Under Primary Dimension, click Other, then Commonly Used, and then select Landing Page. Boom, you will see all landing pages from Google organic.

Dimension by landing page.

But remember, we’re trying to isolate problematic page types. This is where the power of regular expressions comes in handy (as mentioned earlier). Unlike GSC, Google Analytics natively supports regex in the advanced search box, so dust off those regex skills and go to town.

Let’s say all of the problematic page types have two sets of five-digit numbers in the url. They aren’t in a specific directory, but both five-digit sequences do show up in all of the urls for the problematic page type separated by a slash. By entering a regular expression that captures that formula, you can filter your report to return just those pages.

For this example, you could use a regex like:
\d{5}.\d{5}

That will capture any url that contains five digits, any character after that (like a slash), and then five more digits. Now all I need to do is export the urls from GA (or just document the number of urls that were returned). Remember, we’re just trying to estimate how many of those pages are indexed, ranking and/or driving traffic from Google. The benefit of exporting is that you can send them through to your dev team so they can further investigate the urls that are getting indexed by mistake.

Filtering landing pages by regex in Google Analytics

Note, you can also use Analytics Edge to bulk export all of your landing pages from Google Analytics (if it’s a large-scale site with tens of thousands (or more) pages in the report. And again, you can combine the Analytics Edge Core Plugin with the GA connector to filter by regex while you are exporting (all in one shot).

Third-party tools like SEMrush
OK, now our case is taking shape. We have the number of pages ranking and driving traffic from Google organic via GSC and GA. Now let’s layer on even more data.

Third-party search visibility tools provide many of the queries and landing pages for each domain that are ranking in Google organic. It’s another great data source for finding pages indexed (since if the pages are ranking, they must be indexed).

You can also surface problematic pages that are ranking well, which can bolster your case. Imagine a thin and/or lower-quality page ranking at the top of the search results for a query, when another page on your site should be there instead. Examples like this can drive change quickly internally. And you can also see rankings over time to isolate when those pages started ranking, which can be helpful when conveying the situation to your dev team, marketing team, CMO, etc.

For example, here’s a query where several pages from the same site are competing in the SERPs. You would definitely want to know this, especially if some of those urls were lower-quality and shouldn’t be indexed. You can also view the change in position during specific times.

Viewing lower-quality pages ranking in the SERPs via SEMrush

For this example, we’ll use one of my favorite SEO tools, SEMrush. Once you fire up SEMrush, just type in the domain name and head to the Organic Research section. Once there, click the Pages report and you’ll see all of the pages that are ranking in Google from that domain (that SEMrush has picked up).

Note, you can only export a limited set of pages based on your account level unless you purchase a custom report. For example, I can export up to 30K urls per report. That may be sufficient for some sites, while other larger-scale sites might need more data. Regardless, you’ll be gaining additional data to play with including the number of pages ranking in Google for documentation purposes (which is really what we want at this stage).

You can also filter urls directly in SEMrush to cut down the number of pages to export, but you can’t use regex in the tool itself. Once you export the landing pages, you can slice and dice in Excel or other tools to isolate the problematic page type.

Query Recipes – Hunting down rough indexing levels via advanced search operators
OK, now we know the number of pages indexed by understanding how many pages are ranking or receiving traffic from Google. But that doesn’t tell us the number of pages indexed that aren’t ranking or driving traffic. Remember, Google takes every page indexed into account when evaluating quality, so it’s important to understand that number.

Advanced query operators can be powerful for roughly surfacing the number of pages indexed that match certain criteria. Depending on your situation, you can use a number of advanced search query operators together to gauge the number of pages indexed. For example, you can create a “query recipe” that surfaces specific types of pages that are indexed.

It’s important to understand that site commands are not perfectly accurate… so you are just trying to get a rough number of the pages indexed by page type. I’ve found advanced search queries like this very helpful when digging into an indexing problem.

So, you might combine a site command with an inurl command to surface pages with a certain parameter or character sequences that are indexed. Or maybe you combine that with an intitle command to include only pages with a certain word or phrase in the title. And you can even combine all of that with text in quotes if you know a page type contains a heading or text segment in the page content. You can definitely get creative here.

If you repeat this process to surface more urls that match a problematic page type, then you can get a rough number of pages indexed. You can’t export the data, but you can get a rough number to add to your total. Again, you are building a case. You don’t need every bit of data.

Here are some examples of what you can do with advanced query operators:

Site command + inurl:
site:domain.com inurl:12/2017
site:domain.com inurl:pid=

Site command + inurl + intitle
site:domain.com inurl:a5000 intitle:archive
site:domain.com inurl:tbio intitle:author

Site command + inurl + intitle + text in quotes
site:domain.com inurl:c700 intitle:archive “celebrity news”

Using advanced query operators enable you to gain a rough estimate of the number of pages. You can jot down the number of pages returned for each query as you run multiple searches. Note, you might need to run several advanced queries to hunt down problematic page types across a site. It can be a bit time-consuming, and you might get flagged by Google a few times (by being put in a “search timeout”), but they can be helpful:

Using advanced query operators to hunt down low-quality pages that are indexed.

Summary – Using The Data To Build Your Case
We started by surfacing a problematic page type that was supposed to be canonicalized to other pages, but was being indexed instead (since the pages didn’t contain equivalent content). Google just wasn’t taking the hint. So, we decided to hunt down that page type to estimate how many of those urls were indexed to make a case for prioritizing the problem.

Between GSC, GA, SEMrush, and advanced query operators, we can roughly understand the number of pages that are indexed, while also knowing if some are ranking well in Google and driving traffic. In the real-world case I just worked on, we found over 35K pages that were lower-quality and indexed. Now my client is addressing the situation.

By collecting the necessary data (even if some of it is rough), you can tell a compelling story about how a certain page type could be impacting a site quality-wise. Then it’s important to address that situation correctly over the long-term.

I’m sure there are several other ways and tools to help with understanding an indexing problem, but this process has worked well for me (especially when you want to quickly estimate the numbers). So, if you ever run into a similar situation, I hope you find this process helpful. Remember, rel canonical is just a hint… and Google can make its own decisions. And that can lead to some interesting situations SEO-wise. It’s important to keep that in mind.

GG

Google’s Core Algorithm Updates and The Power of User Studies: How Real Feedback From Real People Can Help Site Owners Surface Website Quality Problems (And More)

By Leave a Comment

Google just rolled out another broad core algorithm update on June 3 (which was preannounced by Google’s Danny Sullivan.) And once again, the core ranking update was big. It wasn’t long before you could see significant impact from the update across sites, categories, and countries. Some sites surged, while others dropped off a cliff. And that’s par for the course with Google’s core updates.

For example, here are three examples of drops from the June 2019 Google Core Update:

But I’m not here to specifically cover the June update. Instead, I’m here to cover an extremely important topic related to all broad core ranking updates – conducting user studies. It’s something I have mentioned in a number of my posts about major algorithm updates, and Googlers have mentioned it too by the way. More on that soon.

My post today will cover the power of user studies as they relate to core ranking updates, and provide feedback from an actual user study I just conducted for a site impacted by several major updates. By the end of the post, I think you’ll understand the value of a user study, and especially how it ties to Google’s core updates by gaining feedback from real people in your target audience.  

Google: Take A Step Back And Get Real Feedback From Real People:
After core updates roll out, Google’s John Mueller is typically pummeled with questions about how to recover, which factors should be addressed to turn things around, etc. And as I’ve documented many times in my posts about core updates, there’s never one smoking gun for sites negatively impacted. Instead, there’s typically a battery of smoking guns. John has explained this point many times over the years and it’s incredibly important to understand.

But beyond just taking a step back and surfacing all potential quality problems, John has explained another important point. He has explained that site owners should gain objective feedback from real users. And I’m not referring to your spouse, children, coworkers, top customers, etc. I’m talking about feedback from objective third parties. i.e. People that don’t know your site, business, or you before visiting the site.

When you conduct a study like that, you can learn amazing things. Sure, some of the feedback will not make you happy and will be hard to take… but that’s the point. Figure out what real people think of your site, the user experience, the ad situation, the content, the writers, etc. and then form a plan of attack for improving the site. It’s tough love for SEO.

Here is one video of John explaining that site owners should gain feedback from objective third-parties (at 13:46 in the video). Note, it’s one of several where John explains this:

Conducting User Studies Through The Lens Of Google’s Core Updates:
When you decide to conduct a user study in order to truly understand how real people feel about a site, it’s important to cover your bases. But it can be a daunting task to sit back and try to craft questions and tasks for people that will capture how they feel about a number of core site aspects. As I explained above, you want to learn how people really feel about your content-quality, the writers, the user experience, the advertising situation, trust-levels with the site, and more. So, crafting the right questions is important.

But where do you even begin??

Well, what if Google itself actually crafted some questions for you? Wouldn’t that make the first user study a lot easier? Well, they have created a list of questions… 23 of them to be exact. And they did that in 2011 when medieval Panda roamed the web.

The list of questions crafted by Amit Singhal in the blog post titled More guidance on building high-quality sites provides a great foundation for your first user study related to Google’s core algorithm updates.

Google’s blog post from 2011 containing 23 Panda questions.

For example, the questions include:

  • Would you trust the information presented in this article?
  • Is this article written by an expert or enthusiast who knows the topic well, or is it more shallow in nature?
  • Would you be comfortable giving your credit card information to this site?
  • Does the article provide original content or information, original reporting, original research, or original analysis?
  • Does the page provide substantial value when compared to other pages in the search results?
  • How much quality control is done on content?
  • And more…

As you can see, these are incredibly important questions to review. The questions can absolutely help you better understand how real users are experiencing your site, how they feel about your site, and ultimately, the questions can help craft a remediation plan covering what you need to change or improve on your own site.

I have used these questions (or variations of them) to run both quick and dirty user studies, and formal studies. The feedback you can receive is absolutely gold. Not just gold, but SEO gold in the age of broad core ranking updates.

Let’s face it, this is exactly the type of information that Google is trying to evaluate algorithmically. So, although it’s not easy to run user studies, and it can be time-consuming and tedious, it’s one of the most important things you can do as a site owner.

Beyond The 23 Panda Questions, More Ideas From The Quality Rater Guidelines (QRG)
The Panda questions provide a great foundation, but you can absolutely run more user testing using Google’s Quality Rater Guidelines (QRG) as your foundation. And there are a boatload of topics, ideas, and questions sitting in the 166-page guide that Google uses with its own quality raters.

Google’s Quality Rater Guidelines (as of May 2019)

I won’t spend too much time on the QRG in this post, but some topics you might test include:

  • Page quality.
  • Needs-met ratings.
  • Usability.
  • Monetization and Advertising.
  • Authors, writers, review boards.
  • Site reputation.
  • E-A-T.
  • User intent.
  • And more…

Now, you can just trust me (and John) and think that user testing is important, or you might want more information. For example, like seeing examples of what you can really learn from a user study. Well, I’ve got you covered. I just conducted a user study for a site that was heavily impacted by the March core update (and that has seen major volatility during several core updates over the years). The feedback we received from the user study was awesome and I’m going to share some of it with you (without revealing the site). I think you’ll get the power of user studies pretty quickly.

User Testing Results: What You Can Learn From Real People: Health/Medical Case Study
Again, the site has seen big swings (up and down) during various core updates and I’ve been helping them identify all potential quality problems across the site (including content-quality, technical SEO, user experience, advertising situation, site reputation, UX barriers, and more).

After fully auditing the site, I used the Panda questions mentioned earlier as the foundation for the user study and tailored some of those questions for the niche and site. Below, I’ll provide some of things we learned that I thought were extremely important for my client to understand. Remember, this is real feedback from real people.

Test-wise, I not only used multiple choice questions, but I also used open-ended questions to learn more about how each user felt about certain situations. In addition, I used a platform that provides session recordings of each user going through the study. For this study I used usertesting.com and I’ll explain more about testing platforms later in this post.

I can tell you that watching and listening to people experience a site is absolutely fascinating. There is so much you can learn from hearing the reaction of users, picking up things they say, and watching how they navigate a site or page.

So, the combination of quantitative feedback, qualitative feedback, and viewing recorded sessions provides the ultimate recipe for surfacing potential problems on a site. And that feedback can directly help site owners craft a remediation plan that goes beyond fixing minor issues. Instead, you can start to address deeper issues and problems. And that’s exactly what Google’s core updates are about… Google is evaluating a site overall and not just looking at one or two factors. Remember, there’s never one smoking gun.

The combination of quantitative and qualitative feedback,
plus recorded user sessions = SEO Gold.

First, some quick background information about the user study:
By the time I was setting up the test, I had already fully analyzed the site and provided many areas for improvement. But, we wanted to gain feedback from real users in the site’s target audience about a number of important topics. Also, I wanted to use the 23 Panda questions as a foundation for the test.

Audience Selection:
Since usertesting.com has a panel of over one million people, I was able to select specific demographic information that enabled us to make sure the test participants were part of my client’s target audience. For example, I was able to select gender, age, household income, if they were parents (and how old their children were), job status, web expertise, and more. I’ll cover more about this later.

Selecting an audience is powerful within usertesting.com’s test creation process.

So, what were some things I wanted to learn from the participants? Here are a few of the things I was interested in:

  • Did users trust the information provided in several articles I asked them to read?
  • Did they think the articles were written by experts, or just people heavily interested in a topic?
  • Was the content original? Or did they think it could easily be found elsewhere on the web?
  • Did they recognize the brand? How about the founders and writers?
  • How did they feel about recency, original publication dates, if the articles were updated, and how that was treated on the page?
  • I asked them to review and provide feedback about the background and experience of the site owners, authors, and the medical review board.
  • I wanted to know if the participants thought there was an aggressive, disruptive, or deceptive advertising situation (since this was a problem when I first started analyzing the site).
  • And more… there were 39 different questions and tasks I had the participants go through.

Below, I’ll cover some pieces of feedback that we thought were extremely helpful. By the way, some of the responses (and video clips) were eye-opening. I’ll provide the details below.

Examples of feedback from the user study (in no specific order):

  • Balance – Several participants mentioned the importance of balance in the article. For example, thoroughly covering the benefits AND risks of certain topics. Again, this is something that can be very important in articles, especially YMYL articles.
  • Triggers – I learned that certain words were triggers for some people, which I could only hear in the video clips. I would have never known that from multiple choice questions. For example, when certain words were read aloud, some participants would react in a way that clearly showed how they felt about that topic. They even said, “whenever I read {enter word here}, that immediately throws up a red flag for me”. Wow, amazing feedback for the site owners.
  • Sources and Credibility – Along the same lines, the sources and citations were extremely important for some of the participants. Some explained that if they see Wikipedia as a source, they immediately become skeptical. One even said it discredits the article. For example, one user said, “wait, so it’s reviewed by a doctor, but it cites Wikipedia… not sure I trust this article at all.”
  • Trust & Reactions – When asked about if a certain participant trusted one of the articles, she laughed out loud. Again, hearing people in the video is incredibly powerful. And laughing is typically not a good thing for a YMYL site. :)
  • Publish Dates – There were several important pieces of feedback regarding publish dates, updated dates, etc. First, some assumed that if there was an updated date on the article, then that meant the entire article had been fully reviewed again. That can be deceptive, since the articles just had specific pieces updated.
  • More About Publish Dates – Some participants absolutely wanted to see the original publish date and the updated date. They did not just want the updated date, since that makes them search for clues about when the article was originally published. Some participants explained the process they go through to find the original publish date, which included checking the sources being cited (and the dates associated with those sources). And then they use a savvy approach of checking the comments for dates.
  • Social Proof – I heard one participant explain how if she sees a lot of comments, then that means it must be a popular website. Very interesting… Comments are tough for many sites due to the onslaught of spam, the time involved in moderating comments, etc., but they do seem important for some people.
  • Author Expertise – Several participants wanted to know the background of the writers as they were reading each article. Since the articles they were reading covered health topics, they immediately went into “skeptical mode”. This was important to see and underscores the importance of having experts write the content.
  • Citing Sources – Several participants explained that just a link to a source wasn’t enough for some articles. They wanted to see stats and facts backing up some claims (in the article itself). For example, maybe providing some of the data directly in the article versus just linking out to another article.
  • “Just A Blog…” – I heard several remarks comparing blogs to medical websites. For the health niche, this was very interesting feedback. There was a negative stigma with blogs for some users, especially for health/medical topics.
  • Advertising Situation – Advertising-wise, there were also some interesting pieces of feedback. Remember, there was an aggressive advertising situation when I first started helping the client, so I was extremely interested in hearing what the participants thought of the current ad situation (which has improved, but the site owners haven’t moved as far as I would like them to). I heard one user literally counting the number of ads as she scrolled down the page. 1, 2, 3, wait more, 4, 5. But in a strange twist, she then said the ad situation was fine… She knew there were a number of ads, but didn’t find them distracting. It’s extremely important to make sure the advertising situation is ok, since Google has explained that aggressive ads can impact a site algorithmically over time.
  • Affiliate Marketing – Regarding affiliate links, I did hear, “Are they just trying to sell me something?? Ok, they probably are…” This is something I have brought up to my client during the audit and it’s a tough conversation to have. But remember, Google has explained that there’s a fine balance when delving into affiliate links or affiliate marketing in general. There must be a lot of value added versus monetization. If the scale tips in the wrong direction, bad things can happen Google-wise. So this piece of feedback was extremely important to see/hear directly from users.
  • Author Expertise – When asked about the expertise of the author of an article, the user started scrolling to find the author information and then said, “Wait, it’s a blog… no, I don’t trust the author at all.” I heard this type of comment several times during the user study. More about building a brand and credibility soon.
  • Content-Quality – When asked about original content across the articles, almost all of the users in the study said there was some original content, but some of it could easily be found in other places across the web. Not one person said the content was original. This underscores the importance of tackling subject matter where you can provide original content, ideas, perspectives, etc. If you write about what many others are writing about, the content can be viewed as quasi-original. That’s not good enough for a tough niche.  
  • Content Value – When asked about substantial value from the content compared to other articles on the topic, every one of the users said it was average compared to the others. You clearly don’t want to strive for “average”. You want 10X content. This was great for my client to see. They have strong articles overall, but users saw them as average compared to the competition.
  • Side note: SERP UX – When watching users go to Google and look for a competing article, it was fascinating to see several scroll right by the featured snippet and select something a little farther down the page (in the standard organic results). Sure, this isn’t a large sample size, but just an interesting side note.
  • Site Design – When researching other articles on a topic, a user commented that all the sites look the same. And those sites ranged from some of the top health sites on the web to academic sites to health blogs. Site design, branding, etc. comes into play here and it’s something that I don’t think many focus on enough.
  • Brand Recognition – Regarding brand, every one of the users in the study said they never heard of the site, brand, etc. This is clearly a signal that the site owners need to work on branding. For example, getting the brand out there more via PR, reaching eyeballs beyond their core audience, etc.
  • Recency – For health topics, I heard a user explain they definitely want to see more recent articles on a topic. The article they were reading was a few years old and that didn’t seem sufficient for her. Recency seemed important (but it must actually be recent and not just an “updated on XX” tag slapped on the page).
  • Affiliate marketing – More comments about “they are advertising {enter product here}” while reading an article. So yes, users pick up on affiliate links. Again, the value from the article must outweigh the monetization piece.
  • Citing sources – There were positive comments about certain sources that were cited, like consumer reports, a scientific study, etc. For health articles, I saw users in the video checking the sources at the bottom of the page, which could help build credibility.
  • Medical review board – Overall, the users liked that articles were reviewed by a medical review board. I heard this several times while reviewing the recorded sessions of participants reading the articles.
  • Expertise and Credibility – When asked about the expertise and background of the site owners, authors, and medical review board, there were plenty of interesting comments. For example, having a medical review board with various types of doctors, nutritionists, etc. seemed to impress the participants. But I did hear feedback about wanting to see those credentials as quickly as possible on the page. In other words, don’t waste someone’s time. Don’t be too cute. Just provide the most helpful information that builds credibility as quickly as possible.
  • Awards and Accolades – For various awards won, users want a link to see more information about that (or they wanted to see more on the page itself). It’s clearly not good enough in this day and age to simply say you won something. Let’s face it… anyone can say that. They want proof.
  • Trust – When asked if they would be comfortable giving their credit card information to the site, most responded, “I’m not sure I would go that far…” or “No, definitely not”. So, there were clearly some breakdowns with trust and credibility. I saw this throughout various responses in the study. My client has some work to do on that front.
  • UX barriers – I noticed errors pop up twice while reviewing the video clips of users going through the site. If these are legit errors, then that’s extremely helpful and important to see. I passed the screenshots along to my client so their dev team could dig in. It’s just a secondary benefit of user testing (with video recordings of each session).
  • And there were many more findings…

As you can see, between reading their responses, hearing their reactions, and then watching each video session, we gained a ton of amazing feedback from the user study. Some of the feedback was immediately actionable, while other pieces of feedback will take time to address. But overall, this was an incredible process for my client to go through.

User Testing Platforms – Features & User Panel
If you just read the sample of findings above and are excited to conduct your own user study, you might be wondering where to start. Well, there are several important things to consider when preparing to launch a user study. The first is about the platform you will use.

Usertesting.com is probably the most well-known platform for conducting user studies and it’s the one I used for this test. I was extremely impressed with the platform. The functionality is killer and their panel of over one million people is outstanding.

In addition, participants sign a non-disclosure agreement (NDA), which can help reduce the chance of your test getting shared publicly. Some sites wouldn’t care about this, but others would care. For example, I know a number of my clients would NOT want the world knowing they are running a user study focused on trust, quality, advertising situation, etc.

Audience-wise, I was able to select a range of criteria for building our target audience for the user study (as covered earlier). This enabled me to have participants that were closely tied to my client’s target audience. It’s not perfect, but can really help focus your audience.

Functionality-wise, you can easily create multiple choice questions, open-ended questions, etc. You can also use Balanced Flow to send users through two different test flows. This can enable you to test different paths through a site or different customer experiences.

Here are some screenshots from the test creation process:

Pricing-wise, usertesting.com isn’t cheap… but could be well worth the money for companies that want to perform a number of user tests (across a range of actions). Remember, the sky’s the limit with what you can test. For example, site design, usability, features, content-quality, site trust, and more. I was ultra-impressed with usertesting.com.

Beyond usertesting.com, I also looked into UsabilityHub (Google is a client of theirs btw) and userlytics. I have not used these other platforms, but they could be worth looking into since they also have large panels of users and what seems to be strong features.

Closing Tips and Recommendations:
Before ending this post, I wanted to provide some closing tips and recommendations when setting up your first test. I am by no means an expert on user testing, but I have learned some important lessons while crafting tests:

  • First, user testing is not easy. It can be time-consuming and tedious (especially when analyzing the results). Build in enough time to craft your questions and flow, and then enough time for fully analyzing the results. You might be surprised how much time it takes to get it right.
  • For Google’s core updates, you can definitely use the 23 Panda questions as a foundation for your test. You also might take a subset of those questions and then tailor them for a specific niche and site. After that, you can use the Quality Rater Guidelines as a foundation for additional tests.
  • Try to not ask leading questions. It’s very hard to avoid this… but don’t sway the results by leading someone down a certain response path.
  • Session recordings are killer. Make sure you watch each video very carefully. I’ve found you can pick up some interesting and important things while watching and listening to users that are trying to accomplish a task (or just while they are reviewing a site).
  • Take a lot of notes… I had a text editor up and running so I could timestamp important points in the videos. Then it was easy to go back to those clips later on while compiling my results.
  • Try to gain both quantitative and qualitative feedback from users. Sure, multiple choice questions are great and can be quick and easy, but open-ended questions can yield important findings that might not be top-of-mind when crafting your test. And then layer on videos of each session, and you can gain a solid view of how real users view your site, content, and writers.
  • Find the right balance for the number of participants. Usertesting.com recommends up to 15 participants for a test. Don’t overload your test, which can lead to data overkill. Try different numbers of participants over a series of tests to see what yields the most valuable results. For some tests, 5 participants might be enough, while other tests might require 15 (or more).  

Summary – User testing can be a powerful tool for sites impacted by Google’s core ranking updates
Google has explained many times that it is looking at many factors when it comes to broad core ranking updates. That includes content-quality, technical SEO, user experience (UX), advertising situation, E-A-T, and more. Google’s John Mueller has also explained that it’s important to take a step back and objectively analyze your site.

Well, a great way to objectively analyze your site is by conducting user testing. Then you can have objective third-parties go through your site, content, features, etc., and provide real feedback. I’ve found this process to be extremely valuable when helping companies impacted by major algorithm updates since it can surface qualitative feedback that is hard to receive via other means. I recommend trying this out for your own site (even if you haven’t been impacted by core updates). I think you’ll dig the results. Good luck.

GG

Beyond The 1K Limit – How To Bulk Export Data From GSC By Search Appearance Via Analytics Edge (including How-to, Q&A, and FAQ)

By Leave a Comment

Google has been releasing new features in the search results more and more recently that can have a big impact on SERP treatment, click-through rate, and potentially traffic. Three of those features are part of Google’s “best answer carousels” and include Q&A, How-to, and FAQ snippets. Q&A has been live for a while already, while How-to and FAQ were just rolled out during Google I/O. Note, you can read my post about How-to snippets to learn more about how they work, what they look like, etc.

I have several clients heavily using these formats, so I’ve been analyzing their performance via Google Search Console (GSC) recently — via the new enhancement reports and the Performance reporting. For example, once you start marking up pages using Q&A, How-to, or FAQ markup, you will see new reports show up under the Enhancements tab in GSC. And those reports can be very helpful for understanding errors, warnings, and valid pages.

How To Analyze Performance In GSC By Search Feature:
From a clicks, impressions, and CTR standpoint, you can check the Performance report to view your data over time. Note, if you have Discover data, then there will be two reports under Performance. The first will say Search Results and the second will be titled Discover.

Once in the Performance report (or the “Performance in search results” report), the Search Appearance tab enables you to drill into data by specific feature. You can see that the site from earlier has both How-to and Q&A results. If you click each category title, then you will be isolating that search feature in the reporting (i.e. the reports will be filtered by that search feature). So, you can view queries, landing pages, etc. for just Q&A or How-to results. This applies to AMP as well.

The 1K Row Limit In GSC. The Bane Of A Site Owner’s Existence
Filtering the reporting by search feature is powerful, but remember, GSC only provides one thousand results by report in the web UI and you can only export those one thousand results. For smaller sites, that should be fine. But for larger-scale sites with thousands, tens of thousands, or more listings, then the reporting can be extremely limiting.

For situations like that, what’s a site owner to do??

Analytics Edge To The Rescue Again. Exporting Beyond 1K Results By Search Feature:
I’ve written several posts about Analytics Edge before, and it’s still my go-to tool for exporting data in bulk from GSC. It’s a powerful Excel plugin that enables you to quickly and efficiently export bulk data from GSC, GA, and more.

Below, I’ll take you step-by-step through exporting your data by Search Appearance from GSC. If you’re a large-scale site that’s using Q&A, How-to, FAQ, and even AMP, then you’re going to dig this. Let’s jump in.

How to use Analytics Edge to bulk export data by Search Appearance:
Note, this will be a two-phase approach. The first run will enable us to pull all Search Appearance codes for the specific property in GSC. Then the second run will enable us to pull all data by that Search Appearance code.

Phase One:

  1. Download and install the Analytics Edge free or core add-in. There’s a free trial for the core add-in if you wanted to simply test it out. But the free add-in will work as well (just with less functionality). After installing the add-in, you should register it in Excel.
  2. Next, install the Search Console connector by clicking the License/Update button in the menu. You can watch this short video to learn how to install connectors. You can click the Google Search row to pull up the connector details (where you can choose to install that connector).
  3. Once you install Analytics Edge and the Search Console connector, access the options in the Analytics Edge menu at the top of Excel. Click the Google Search drop-down and select Accounts. This is where you will connect Analytics Edge with the Google account(s) you want to download data from. Go through the process of connecting the Google account you want to work with. You can also make one account the default, which will save you time in the future.
  4. Once you connect your account, click Google Search in the Connectors section, and then Search Analytics.
  5. Name your Macro and click OK.
  6. Select an account and then a property from GSC.
  7. We will use a two-phase approach for exporting data by Search Appearance. First, we are going to view the various options we have under Search Appearance (there will be codes that show up representing each SERP feature available to a property). We will use these codes during our second run to pull all data for each specific search feature (like How-to, Q&A, FAQ, AMP, etc.)
  8. Under the Fields tab, select searchAppearance, which will move that option to the Dimensions window.
  9. For the Dates tab, you can leave “Last 3 Months” active (which is the default).
  10. Leave everything else the same and click “Finish”.
  11. Analytics Edge will return all of the possible Search Appearance codes for the site for the time period you selected. For example, in the screenshot below, there were impressions and/or clicks for AMP (article and non-rich results), Q&A, How-To, and others for the property I selected.
  12. Copy the codes from the searchAppearance column to a text file so you can reference them in phase two of our tutorial. You will need these codes to export all data by that specific search feature.

Phase Two:

  1. Now we are going to use the searchAppearance codes to export data in bulk for a specific search feature. Click Google Search again, and then Search Analytics. Choose an account and a property again. When you get to the options screen, select the dimensions you want to export (in the Fields tab). For this example, let’s select query (to see the queries yielding How-to snippets in the search results).
  2. Next, go to the Filters tab and find the Appearance field. In that field, enter TPF_HOWTO, which is the code for How-to snippets. If you want to export data for another search feature, just use that code instead. Exporting how-to snippets from GSC.
  3. Next, select the dates you want to run the report for. For this example, I’ll select “Last 28 days”.
  4. Then under Sort/Count, select clicks and then descending in the “sort by” dropdown. This will sort the table by the queries with the most clicks over the past 28 days (that yield How-to snippets).
  5. Then click “Finish”.
  6. Analytics Edge will run and export all of the queries yielding How-to snippets. This can take a bit of time (from a few seconds to a minute or longer) depending on how large the site is and how much data needs to be exported. Note, just a sample of data will be presented in memory in the worksheet (and highlighted in green). You need to “write to worksheet” to show all of the data.
  7. To do that click File and then “Write Worksheet”. Name the worksheet and click OK. You will now see a new worksheet containing all of your data. For this example, I see 26K+ queries that have yielded How-to snippets over the past 28 days. Yep, over 26K!
  8. Congratulations! You just exported search feature data from GSC and blew by the 1K row limit!

Tracking, Using, and Learning From The Data:
Once you export your data by Search Appearance in bulk, you will have full access to all of the queries yielding Q&A, How-To, FAQ snippets, AMP, and more. You can track their position, double check the SERPs to understand the SERP treatment for each feature, understand the click-through rate for each snippet, and more.

For example, you might find that “list treatment” for How-tos is yielding a higher click through rate than the carousel treatment. Or you might find that a certain How-to has a featured snippet in addition to the How-to snippet (a How-to/featured snippet combo). And then you can check metrics based on that situation. You get the picture!

You can see an example of a How-to/featured snippet combo below:

A How-to/featured snippet combo.

My recommendation is to export data by each search feature for the past 28 days and start digging into the data. Then regularly export the data (like weekly) to understand the changes over time. E.g. changes in metrics, SERP treatment, and more.

Summary – Free Yourself From the 1K Row Limit In GSC By Exporting SERP Feature Data Via Analytics Edge
Now that more and more features are hitting the SERPs, using a tool like Analytics Edge can help you export all of your data, versus just one thousand rows per report. And when you export all of the queries and landing pages per SERP feature, you can glean more insights from the data. If you are using AMP, How-to, Q&A, or FAQs, then I highly recommend exporting your data via a tool like Analytics Edge. I think you’ll dig it.

GG