Archives for March 2017

How To Properly Set Up Pagination With Sorting Parameters Using Rel Next/Prev And Rel Canonical [SEO Tutorial]

March 30, 2017 By Glenn Gabe 20 Comments

Last Updated: October 2021

How to set up pagination using rel next/prev and rel canonical.

Update – October 2021
Google just published new help documentation containing best practices for e-commerce sites, which included a section about pagination. In that section, Google explains that for category pagination, it recommends having each paginated url indexable and that each contains a self-referencing canonical tag. Google also emphasizes to not canonicalize each page in the pagination to the first page in the set. In addition, Google explains to link paginated pages sequentially to help it understand the relationship between pages in the pagination. And as part of that, Google recommends providing a link back to the first page in the pagination on all pages in the collection (which can give Google a hint that the root page in the pagination is the best landing page versus the other deeper pages in the pagination.)

Best practices for pagination from Google

In addition, I also just published a case study showing what happens when 67% of a site’s indexed urls are pagination. The site is using the approach I have documented in this post where all pagination is indexable using self-referencing canonical tags. The case study backs up what Google’s John Mueller has been explaining for a long time (that Google has a lot of experience with pagination, and that it can often just work).

Update – April 2021
Google’s John Mueller just covered pagination in a Search Central Hangout when asked about the best way to handle pagination for categories. First, he said to determine if you are addressing articles split into multiple urls or category pagination (like what’s found on many e-commerce sites). If you are addressing articles that have been split up into multiple urls, then make sure each url is linked to & indexable. For category pagination, if your urls are crosslinked well on the site, then do what makes sense for you. For example, you can noindex the pagination beyond page one, you can canonicalize those urls to page one, or you can have the urls indexable if you want (with self-referencing canonicals). It totally depends on your specific situation. Here is the video from John:

Update – December 2019: Google provides new guidelines and recommendations
Google’s John Mueller just presented at the Webmaster Conference in Tel Aviv on December 4, 2019. In that presentation, he covered how to handle pagination now that rel next/prev is not being used for indexing purposes. It sounds like more information will be coming via a blog post, but I wanted to provide John’s recommendations here until we have more information. Igal Stolpner tweeted a photo from John’s presentation where he was covering how to best handle pagination now that rel next/prev doesn’t help from an indexing standpoint.

As you can see in the tweet below, John explains to link naturally between pages, to use clean urls (with no or few parameters). He also explained that it’s important to determine the goal of the paginated content and then adjust your strategy based on that.

For example, if it’s paginated content (like an article that’s paginated), then you should link all pages and make them all indexable. But if it’s what he calls “detail links”, like for e-commerce sites, then you can either link all pages, or if all content is findable via cross-linking, then you can just have the first page indexable (or the first few pages in the pagination). You can canonicalize the rest to page one or noindex the rest. Again, I hope Google publishes a post soon with more detail about these recommendations. I’m sure many people will have questions. :)

Google's John Mueller provides guidance about setting up pagination.

Update: March 2019: Google just announced that after evaluating its indexing signals, it realized it’s not using rel next/prev as an indexing signal anymore. As of a few days ago, Google was still recommending using rel next/prev to signal to them that pages were part of a set, and that Google should consolidate indexing properties from across that set. But it ends up they haven’t been using rel next/prev for a few years. Crazy, but true.

Google announces that rel next/prev is not an indexing signal anymore.

Confirmation from Google's John Mueller that rel next/prev isn't used for indexing.

But don’t feel like you need to remove rel next/prev or radically change your setup. This is not a recent change on Google’s end. So if you’re setup is working for you, then don’t change a thing. Google’s John Mueller has explained that as well. You can read this post on Search Engine Roundtable by Barry Schwartz, which covers the change. And you can watch Google’s John Mueller explain more about the situation in a recent webmaster hangout video.

Here is the video segment (starting at 14:35 and lasting about ten minutes):

My recommendation is to try and fold paginated articles together into one strong url (or at least cut down on the amounts of pages that each article is broken out to). As Google has explained, each page needs to stand on its own. And for pagination for lists of items (like articles in categories, e-commerce product categories, etc.), pagination should still be used like you are hopefully doing now. And then use the right canonicalization strategy based on your situation. You can read more about that in my article below. Just skip the rel next/prev information and focus on the canonicalization sections.

As Google always says, change in constant. At least they informed us about this change, even if it was a few years late. :)

———————–

While helping companies improve overall quality, I’m typically neck deep in performing technical audits and analyzing crawl data. And with many large-scale sites, it’s not uncommon to come across a heavy amount of pagination used throughout those sites. Now, pagination is totally fine to have and can help both users and bots get to your content through organized categories. But, it’s important that your pagination is set up properly from an SEO perspective, so you can get the most bang for your paginated buck. Unfortunately, I’ve seen so many pagination flaws during my audit travels, it’s not even funny.

Based on the botched setups I have seen, and the confusion surrounding pagination, I’ve been jokingly telling clients that I need to write a blog post about how to properly set up rel next/prev when a pagination uses sorting or filtering. Because if I did, I could just point new clients there versus explaining it over and over. So, I finally decided to write it up. :) And yes, I know there are many posts out there about pagination and SEO. But, I wanted to provide one detailing the exact setup that I recommend (based on Google’s official recommendations and my experience helping many large-scale sites that employ pagination).

This post will cover a number of topics under the pagination umbrella, including how to set up rel next/prev, how to handle sorting and filtering parameters, and how to combine rel canonical with rel next/prev. My hope is that this post can clear up some of the confusion surrounding pagination and SEO.

Your Goal and How Google Should Process The Setup
First, let me quickly explain why this is important. When using pagination, you often have several pages of content within a specific category. For example, pagination is prevalent on ecommerce sites that might have hundreds of products under a certain category. Or you might have sites with reviews that might have dozens or more reviews per product.

From an SEO perspective, it might not sound great to have dozens of category pages indexed by Google. But you absolutely could, as long as you know Google is handling those pages properly. The catch is that you should use rel next/prev in the code of your component pages so Google can consolidate indexing properties from across the paginated set. That means Google can treat your paginated set as one, versus dozens of separate pages containing a list of products. And when Google does that, it can surface the right page from the paginated set in the SERPs based on query, but it will still treat the paginated set as one (signals-wise).

Using rel next/prev to consolidate indexing properties.

Warning: Don’t Canonicalize All Component Pages To The First Page! Set Up Rel Canonical Correctly
Also, and this is important to understand, if you canonicalize all component pages to the first page in your pagination, then Google can’t index any of the additional content. So you are diminishing the power of the paginated set by canonicalizing each component page to the first page. It’s simply not the right setup. I’ll cover how to use rel canonical the right way with pagination soon.

A Note About Google Resources for Pagination and Rel next/prev
Before we begin, I wanted to mention that there are some great resources from Google explaining more about this topic, and I highly recommend checking them out. For example, there’s a great video from former Googler Maile Ohye covering how to use rel next/prev in detail. And then Google has written several blog posts about pagination, including one about common mistakes that site owners make when implementing rel canonical on their sites. There’s a part about not canonicalizing all component pages to the first page in the pagination. Between my post today and those resources, you should be ready to rock and roll pagination-wise. Let’s get started.

How To Properly Set Up Rel Next/Prev with Filtering: Step-by-step
It’s best if I use an example that’s easy to understand, so let’s say we have an ecommerce site named “Glenn’s Baseball Gloves”. Hey, I coach baseball and I’m helping my players choose the right gear now. :) Let’s set up the pagination for a category of pre-broken in baseball gloves. Here we go!

1. Our Ecommerce Category
The category url is https://glennsgloves.com/baseball-gloves/pre-broken-in.htm. The category contains 75 products and we’re going to provide fifteen gloves per page. So, we’ll have five component pages in the pagination. Page two will have the following url: https://glennsgloves.com/baseball-gloves/pre-broken-in.htm?page=2. Then page=3, page=4, etc.

Handling pagination on an ecommerce site.

2. Filtering or Sorting The Pagination
We will also provide sorting for our pagination. To keep this simple, let’s provide sorting by price. So if the user chooses to sort “Price High-To-Low”, the pagination will reorder the gloves and show the most expensive gloves first. The sorting parameter will be price=X, where X could be high or low.

Sorting by price on an ecommerce site.

So we might end up with a url that looks like this:

https://glennsgloves.com/baseball-gloves/pre-broken-in.htm?page=2&price=high

3. Rel next/prev
This is the easy part. You will need to add rel next/prev tags in the head of each component page that provides the next url in the pagination, and the previous. The only exceptions are the first and last pages in the pagination. That’s because the first page will have a rel next tag, but won’t have a previous tag. And the last page will have a rel prev tag, but won’t have a next tag. All other component pages should contain both rel next and prev.

For our root category page https://glennsgloves.com/baseball-gloves/pre-broken-in.htm, we’ll use the following in the head of the document:

Remember, the first page will not contain a rel=”prev” tag since there isn’t a previous page!

And our page=2 url will contain the following tags in the head of the document:

Important note: I’ve seen many sites use ?page=1 in the rel prev tag when pointing back to the root category page, when that’s not the actual url. The url actually doesn’t contain any parameters, so it’s just https://glennsgloves.com/baseball-gloves/pre-broken-in.htm without page=1.

You should complete the tags for the remaining component pages and then include just a rel prev tag on the last page (since there’s not a “next” url). For example, the final page in the pagination (page 5) would contain the following rel prev tag in the head of the document.

4. Rel canonical
When setting up the pagination, I highly recommend using self-referencing canonical tags on each component page. Rel canonical and rel next/prev are separate concepts and can be used together on the same url. Again, these should be self-referencing canonical tags for each component page.

The root page should contain a self-referencing canonical tag, page=2 url should as well, so on and so forth. Remember, we want Google to index each of the pages so it can consolidate indexing properties from across the paginated set. Don’t simply canonicalize all component pages to the first page. That’s not the right setup. Google even lists that in their common mistakes blog post for rel canonical! See screenshot below.

Don’t do this!
Don't canonicalize all component pages to page one in the pagination.

Here is an example of what we should implement based on our fictitious website:

The root category url https://glennsgloves.com/baseball-gloves/pre-broken-in.htm would contain the following rel canonical tag:

and then the second url in our pagination https://glennsgloves.com/baseball-gloves/pre-broken-in.htm?page=2 would contain:

Again, DO NOT canonicalize all component pages to the first page in the pagination. That’s not the proper setup (as documented above)! And rel canoncial should contain the url with the page parameter, but not contain any sorting or filtering parameters. I’ll cover more about that next.

5. Handling Sorting or Filtering In The Pagination
This part seems to confuse a lot of people, just based on the number of parameters being handled and how it ties to rel canonical and rel next/prev. If filtering or sorting parameters are provided, then you should include them in rel next/prev tags, but not include them in rel canonical. Rel canonical should only contain the core url for the component page without the filtering parameters. Google’s Maile Ohye explained this in her video about pagination.

Also, it’s important to keep a consistent order for your parameters. In my opinion, you should always start with the core url and page parameter (page=X). Then layer on filtering parameters, but keep the order consistent. Don’t use page=X&price=Y&size=Z on one component page and then page=X&size=Y&price=Z on the next. Or worse, don’t move the page=X parameter to after the filters! Make it easy for Google to understand the structure so it can consolidate indexing properties without running into trouble.

For our example, let’s say a user sorted by price, the first page would contain the following rel next tag and rel canonical tag:

Then page 2 would contain:

And page 3 would contain:

Notice how rel next prev carries the sorting parameters from one url to the next, while rel canonical only references the core url. We would follow the same formula for page four. And the final page (which is page 5 in our example) would contain the following tags. Remember, the final page would not contain a rel next tag since there isn’t another page in the pagination:

Rel=Got It?
And that’s it! Now you are handling rel next/prev correctly on each component page of the pagination, as well as rel canonical. And you are also providing consistent sorting parameters via rel next/prev. And you’re not canonicalizing all component pages to the first page (which is the wrong way to go).

Now Google can understand the structure of your pagination, consolidate indexing properties from across the paginated set, treat the pagination as one signals-wise, and then surface the correct url in the SERPs based on query. Groovy. :)

A Note About Indexation
You will probably see indexation increase when checking the Index Status report in Google Search Console (GSC) as Google crawls and indexes the component pages in your pagination. This is especially true for those sites that were once canonicalizing all component pages to the first page in the pagination. This is totally normal to see, so don’t freak out if it starts to rise. Indexation should level off as Google indexes all of your component pages.

Indexation will increase when using rel next/prev correctly.

Summary – Consolidate Indexing Properties With The Right Pagination Setup
The next time you’re faced with providing a heavy amount of pagination by category, you can use the setup I covered in this post. It’s based on Google’s official recommendations and it’s something I have helped clients with many times in the past. So don’t fall victim to common pagination problems. Instead, let Google crawl and index all of your component pages, but tie them together via rel next/prev. And then use rel canonical to send clear signals about which pages should be indexed. Pagination shouldn’t be scary and I hope this post made the setup a little less intimidating… Now paginate away!

How To Export All Search Queries From Google Search Console To Compare Clicks And Impressions After An Algorithm Update (using Analytics Edge)

March 18, 2017 By Glenn Gabe 8 Comments

How to export queries from GSC via Analytics Edge.

When experiencing a traffic drop due to an algorithm update, redesign, migration, or some other event, it’s important to dig into the drop to understand the queries and landing pages that saw the biggest change. When you do, you can have a strong feel for the pages impacted and the queries leading to those pages (and that can sometimes help you begin to identify the cause of the drop).

When analyzing a traffic drop or surge, there’s nothing better than your own data. For example, digging into Google Search Console and Google Analytics – or whichever analytics package you are using. For example, with GSC’s Search Analytics reporting, you can view clicks, impressions, and average position for queries and landing pages, while also comparing timeframe to see the change. That’s important and can quickly help you gauge the big winners or losers.

In previous posts, I’ve mentioned my tutorial about creating a Panda report, which can help you identify landing pages that saw the biggest change when experiencing a traffic drop. That’s based on Google Analytics data and compares sessions from Google organic after the hit to before the hit (in Excel). It’s a great report that can help surface urls seeing the biggest movement.

But, it doesn’t provide the queries seeing the biggest drop. And that’s exactly what we’re going to tackle today.

The GSC UI Limit and The Power of APIs
If you’re working in GSC a lot, then you know the major limitation with its UI reporting. I’m referring to the one thousand row limit in GSC’s Search Analytics reporting. For larger sites, this is extremely frustrating and can inhibit the amount of data you can work with and export.

The dreaded 1k row limit in GSC.

Thankfully, there’s an API that can enable those with developer resources to pull all of the data. That’s great, but what if you don’t have dev resources readily available? How can you get that data??

Analytics Edge to the rescue (again)!
Last year I wrote a post explaining how to export all of your landing pages from GSC via Analytics Edge. It’s an awesome solution that quickly and efficiently pulls GSC data via the API. You can read my post to learn more about that.

For today’s post, I will explain how to go even further by exporting all queries from two different date ranges, and then using the power of Excel to identify the difference in clicks and impressions. Again, after a traffic drop due to an algorithm update, this is invaluable data to have.

Let’s begin.

Step-by-step: How to compare clicks and impressions for queries in Excel:

1. Download and install the Analytics Edge free or core add-in. There’s a free trial for the core add-in if you wanted to simply test it out. But the free add-in will work as well (just with less functionality). After installing the add-in, you will register it in Excel.

2. Next, install the Search Console connector by clicking the Manage Connectors button in the menu.

Manage connectors in GSC

3. Once you install Analytics Edge and the Search Console connector, access the options in the Analytics Edge menu at the top of Excel. Click the Google Search drop-down and select Accounts. This is where you will connect Analytics Edge with the Google account(s) you want to download data from. Go through the process of connecting the Google account you want to work with.

Manage accounts in Analytics Edge.

4. Now that you’ve connected a Google account to Analytics Edge, click the Google Search drop-down again and select Search Analytics. This will start the wizard for working with the Search Analytics reporting from GSC. Name the macro whatever you want, and you’ll be taken to the next screen.

5. Now pick an account to work with and a specific website. You should see all of the sites you have access to in the list. Also, select the query dimension, since we want to export all queries from GSC. Next you can select filters and the date range.

Select queries in Analytics Edge via Fields tab.

6. There are three tabs in the report interface. The first is labeled Fields, and we just set that up by selecting the query dimension. If you click the Filters tab, you can filter your report by query, page, country, device, type, and search appearance. You can also limit the number of rows you want to export. For our purposes, I would leave them all blank (but you can focus your export by using these filters in the future.)

Filter queries in Analytics Edge.

7. The third tab, labeled Dates, enables you to select the date range for your report. This is extremely important since we want to compare two different time ranges (due to an algorithm update or some other event that caused a traffic change.) For the start date, choose the date the traffic drop began and then choose an appropriate end date. Make sure to give yourself enough days to work with or the comparison might not contain enough data. Then click Finish to start the export.

Select date range in Analytics Edge.

8. Analytics Edge will export your results, but it will only show a sample of the export in the worksheet highlighted in green (in memory). If all looks good with the sample, then you must “write to worksheet” to see all of the data that was exported. In order to do this, click File, and then Write Worksheet. Simply enter a name for your new worksheet like “Current” and click Finish. Boom, check the new worksheet that was created to find all of your queries exported from GSC.

Write To Worksheet in Analytics Edge.

If you followed the instructions above, then you are staring at glorious query data from the timeframe you selected. Not a one-thousand row sample of that data, but the whole enchilada. For example, you can see below that I exported close to 39K queries in just a minute or two. Awesome, right?

Export all queries from GSC via Analytics Edge.

9. Repeat step seven again but this time pull data from the previous timeframe. This is the data we are going to compare against. Make sure the amount of days matches what you pulled in the previous step. For example, compare three weeks of data to three weeks of data, or four to four. Don’t compare three weeks to one week or it’s not a true comparison. Also, if your site does not have consistent traffic all week, then make sure your comparison matches the days of the week (versus simply checking the previous X days). You can name this worksheet “Previous” to keep it simple.

Excel Magic Through Vlookup
Once you export the previous timeframe, you will have two worksheets with query data. This is where the magic of Excel comes in handy. We’re going to use vlookup to check the previous timeframe for a query, and if it matches a query in the current timeframe, then we’re going to pull clicks and impressions into our current worksheet. That will enable you to compare metrics for each query prior to the traffic drop, to after the drop.

10. In the worksheet named “Current”, create a new column titled “Clicks Previous”. This is where we will pull click data from the worksheet containing metrics from the previous timeframe. In that field, enter the following formula:

=VLOOKUP(A2,Previous!A:E,2, FALSE)

Using vlookup to access clicks from previous timeframe.

How it works: vlookup will check the query in the current worksheet and then cross reference the query in the second worksheet (which contains data from the previous timeframe). If the query is present, it will pull the clicks data from the second worksheet into the first.

A2 is the column that contains the query in our “Current” worksheet.
Previous is the name of our second worksheet containing data from the previous timeframe. Note, if you named the worksheet more than one word, you’ll need to add single quotes around the text.
A:E is an array that tells Excel which columns to use for the lookup in the second worksheet (Previous). We want to cover all of the columns, which for me contains data in column A through E.
2 is the column number (by order) we want to cross reference in the Previous worksheet. Count the columns from left to right and you’ll see “clicks” is the second column over. So that’s why we use 2 as the number in the formula for the fourth argument.
False is used as the Range_lookup because we want an exact match for the query.

11. I recommend testing the formula on one cell that you know contains a query that’s present in both the Current and Previous worksheets. If the formula worked, then you should see a value show up in the field for Clicks Previous. Quickly check the Previous worksheet to make sure that’s accurate and you’re good to go.

12. Now you need to copy that formula to all rows in the “Clicks Previous” column. The easiest way to do that is to single click the cell containing the new formula and then hover your cursor over the bottom right corner of the cell. Then double-click the small green square. The formula will be copied to all remaining cells in the column. And you should see the previous click count show up in each cell.

13. Repeat the process above for impressions. Create a new column titled “Impressions Previous” and use the same vlookup formula. However, you’ll need to change the column you are using for the lookup. The formula should look like this:

=VLOOKUP(A2,Previous!A:E,3, FALSE)

Using vlookup to import impressions from previous timeframe.

I bolded the 3 since that’s the only difference in the formula. That tells Excel to use the third column for the lookup in our “Previous” worksheet. That third column contains impressions.

14. If all looks good, then copy that formula to the rest of the rows in the “Impressions Previous” column by clicking the small green square in the bottom right corner of the cell again.

15. Now you will create additional columns that simply show the difference in clicks and impressions between the previous and current timeframe. Create new columns titled “Clicks Difference” and “Impressions Difference”. Then enter the following formula which will subtract the previous timeframe metric from the current timeframe:

=B2-F2

Note, B2 is the column containing clicks from the current timeframe and F2 contains clicks from the previous timeframe.

Calculating the difference between clicks.

Then repeat for impressions, but this time using the following formula:

=C2-G2

Note, C2 contains impressions from the current timeframe while G2 contains impressions from the previous timeframe.

To copy those formulas to the rest of the rows in each column, use the trick I explained earlier to single click the cell you want to copy and then hover your cursor over the bottom right corner. Then double click the small green square.

At this point, you should have a worksheet that provides current metrics from GSC along with metrics from a previous timeframe. And you should also see the difference in clicks and impressions by query. This data is invaluable for digging into queries that dropped based on an algorithm update, redesign, migration, or other event that caused a traffic drop. Awesome, right?

Some Final Tips:

You can use conditional formatting to highlight cells that dropped by a certain amount. That can help you and your team quickly identify queries that saw the most impact.
You can add a column for “Position Change” and use average position in your vlookup. Then you can see the number of positions each query dropped or gained.
You can follow my tutorial about using Analytics Edge to export all landing pages to dig into the pages that dropped as well. Then you can run this report again and use landing pages versus queries. Then you can dig into the specific urls that dropped the most.

Summary – Use the API, Love the API
Google Search Console (GSC) is a killer tool that every SEO should use, but the one thousand row limit can be maddening for site owners of larger websites. Using the API via a tool like Analytics Edge can enable site owners to export well beyond the one thousand row limit. And then you can use the power of Excel to analyze the data. It’s a great combination that I’ve found incredibly valuable while researching traffic drops due to algorithm updates (or other events). I recommend trying this out today. I have a feeling you are going to dig it.