Internet Marketing Driver

  • GSQi Home
  • About Glenn Gabe
  • SEO Services
    • Algorithm Update Recovery
    • Technical SEO Audits
    • Website Redesigns and Site Migrations
    • SEO Training
  • Blog
    • Web Stories
  • Contact GSQi

LiveHttpHeaders and SEO, How to Check Your HTTP Response Headers for Red Flags

April 11, 2008 By Glenn Gabe

Share
Tweet
Share
Email

Using LiveHttpHeaders to Check Your HTTP Response Codes for SEODoes your website throw a 302 when it should throw a 301? Does it throw a 200 when it should be a 404? Are there 500’s thrown on your site that look like 404’s? Do you think I’m insane yet? Hear me out…

Whether you understand the introduction above or don’t know what I’m talking about, there’s still something extremely important for you in this post. Every time you load a webpage, your browser REQUESTS a file and then the server provides a RESPONSE to that request (also called a Response Header). Response headers can help you identify critical issues on your site (especially from an SEO standpoint). Now, you probably have a few key questions.

1) How do I check my response headers?

2) What should I be looking for?

Although I can’t cover everything about response headers in this post, I will answer the two questions listed above and provide some examples along the way.

Let me start by answering the first question since it’s the easiest… I highly recommend using LiveHttpHeaders, an add-on for Firefox that displays http headers in real time (as you browse webpages). This tool can save you a lot of time and possibly help you diagnose some serious SEO-related issues. I will answer the second question later in the post.

Install LiveHttpHeaders Now:

First, visit the LiveHttpHeaders project website and install the add-on. You will need to restart Firefox after installing LiveHttpHeaders. Once restarted, you can trigger LiveHttpHeaders in two ways. You can click Tools, LiveHttpHeaders, which will trigger a new window where you can view header responses in real time as your browse the web. You can also click View, Sidebar, LiveHttpHeaders to view response headers in a sidebar within Firefox. I prefer the new window, since I have dual monitors and it doesn’t take up any browser space. :-) Either way works fine.

To quickly test it out before we go any further, go and visit Google with LiveHttpHeaders running. When you hit the homepage of Google, you will see a bunch of information scroll by in LiveHttpHeaders. For our purposes, let’s look at the top of the window (the first piece of information sent back to you). I have stripped out some of the information you don’t need to focus on for this example.

http://www.google.com/

GET / HTTP/1.1

Host: www.google.com

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11

Keep-Alive: 300

Connection: keep-alive

HTTP/1.x 200 OK

Content-Type: text/html; charset=UTF-8

Content-Length: 2547

I have bolded the response code, which shows 200 (or ok). 200 is a good thing… and now I can explain more about response headers and codes. By the way, where you see 200 in the window is where you would also see other response codes like 301, 302, 404, etc.

Back to our two questions for a minute. The second question was “What should I be looking for?” In a nutshell, you should be looking for the file requested and the response code sent by the server. Let’s start with a definition of a response header and identify some core http response codes.

As I mentioned earlier, when you load a webpage, two things happen. There is an http request (by your browser) and then an http response is sent (by the server). There is a response code returned as part of the http response. Some of the following codes might be familiar to you, and others aren’t. If you focus on SEO, then get to know them…they can really help you diagnose problems on your website.

Some of the most common http response codes are:

200 – ok (the webpage was returned ok)

301 – Permanent redirect (seo friendly) :)

302 – Temporary redirect (don’t use this unless you absolutely have to!)

400 – Bad request (uh, not good)

401 – Unauthorized (you need to be authenticated)

403 – Forbidden (doesn’t matter who you are, it’s forbidden)

404 – Not found (not necessarily a bad thing…I’ll explain more later.)

500 – Internal server error (something went very wrong processing the webpage…)

502 – Bad gateway (also not a good thing)

Why are http response codes important? One of your goals as an SEO is to enable the bots to easily index your site. You don’t want them to get caught up in any way, shape or form. For example, 302 redirects are not the SEO-friendly way to tell Google where a page you removed now resides (you should use a 301 redirect instead). So constantly providing 302’s would be a very bad thing to do. Or how about throwing a 200 (ok) when you really should be throwing a 404. For example, the page isn’t there, but you just told Google that it is. Again, not a good thing to do. Therefore, finding 302’s, 404’s, 403’s, 500’s, etc. is critical to creating a clean path for the bots, which means you can have more of your content indexed and at a solid frequency. Let’s take a look at how LiveHttpHeaders can help you out.

Checking Your Response Headers:

Let’s take a look at a hypothetical situation. One morning you wake up and decide that you want to increase your natural search rankings. You launch an SEO initiative and get moving quickly. The first thing you want to do is to audit your current site structure (since you know that without a sound and clean structure, you’re dead in the water). So as part of your audit, you want to ensure your response headers and response codes look ok.

You fire up LiveHttpHeaders and visit your website:

* You hit the Homepage, 200 returned,

* You visit a Top Level Product Category, 200 returned,

* Then you try and visit a product detail page and you hit a 302 redirect. Hold on… You find that all links to your product detail page go through a 302 redirect. This was implemented as part of a recent code change. This is something you would want to change ASAP. The content on your product detail page is obviously important so you wouldn’t want to be throwing 302’s prior to the bots hitting those pages…

* But it doesn’t stop there. You know that you changed dozens of older product pages recently and created new URL’s. You check out the old URL’s and find 302 redirects to the new product pages. You would want to change that too… and provide a 301 redirect from the old page to the new page, safely passing link power from the old page to the new one.

* Then you check out some product categories on your site that have been removed completely (you won’t be selling those products anymore), but you find 200’s instead of 404’s. A 404 (page not found) is the proper response code to throw in this situation, as it will tell the engines that the page has been removed and that it should be de-indexed. You don’t want the page to be indexed if it’s not actually there, right?

* Last you check some newly added pages and find they are not displaying correctly. It looks like they aren’t on the site, which means you should see a 404. But…the server returns a 500 (or internal server error). Again, not a good thing as the bots traverse your website content and this is something invisible to the naked eye as you test your website. You would need to be checking response codes to find this issue…

OK, I think you get the picture! Keep in mind that a full SEO assessment covers much more than just checking response codes, but it’s an important part to revealing SEO-related issues. And you know what? Sometimes it’s darn easy to find a serious issue that can be resolved fairly quickly. For example, providing a 302 redirect right on your homepage! Or throwing 200’s for any page that’s been removed from your site.

Think Like a Bot:

I’ll end this post with an analogy. Imagine you needed to check every room in a 10 story hotel to document the type of TV that’s in each room. But the elevators don’t work properly, some of the staircase entrances are locked, and every now and then the room numbers change on you. Would you have an easy time completing your task? Would you keep trying to come back to “index” each room? Or would you stop a few rooms in and say, “Hey look, Lost is on.” And then sit back and watch the show….and forget about the TV’s (or your content). ;-)

GG

Share
Tweet
Share
Email

Filed Under: SEO

Connect with Glenn Gabe today!

RSS Latest Blog Posts

  • Continuous Scroll And The GSC Void: Did The Launch Of Continuous Scroll In Google’s Desktop Search Results Impact Impressions And Clicks? [Study]
  • How to analyze the impact of continuous scroll in Google’s desktop search results using Analytics Edge and the GSC API
  • Percent Human: A list of tools for detecting lower-quality AI content
  • True Destination – Demystifying the confusing, but often accurate, true destination url for redirects in Google Search Console’s coverage reporting
  • Google’s September 2022 Broad Core Product Reviews Update (BCPRU) – The complexity and confusion when major algorithm updates overlap
  • Google Multisearch – Exploring how “Searching outside the box” is being tracked in Google Search Console (GSC) and Google Analytics (GA)
  • Sitebulb Server – Technical Tips And Tricks For Setting Up A Powerful DIY Enterprise Crawler (On A Budget)
  • Google’s Helpful Content Update Introduces A New Site-wide Ranking Signal Targeting “Search engine-first Content”, and It’s Always Running
  • The Google May 2022 Broad Core Update – 5 micro-case studies that once again underscore the complexity of broad core algorithm updates
  • Amazing Search Experiments and New SERP Features In Google Land (2022 Edition)

Web Stories

  • Google's December 2021 Product Reviews Update - Key Findings
  • Google's April 2021 Product Reviews Update - Key Points For Site Owners and Affiliate Marketers
  • Google's New Page Experience Signal
  • Google's Disqus Indexing Bug
  • Learn more about Web Stories developed by Glenn Gabe

Archives

  • July 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • June 2009
  • May 2009
  • April 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • September 2008
  • August 2008
  • July 2008
  • June 2008
  • April 2008
  • March 2008
  • February 2008
  • January 2008
  • December 2007
  • November 2007
  • October 2007
  • September 2007
  • August 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • November 2006
  • October 2006
  • GSQi Home
  • About Glenn Gabe
  • SEO Services
  • Blog
  • Contact GSQi
Copyright © 2023 G-Squared Interactive LLC. All Rights Reserved. | Privacy Policy
This website uses cookies to improve your experience. Are you ok with this? You can always opt-out at a later time if you wish. Cookie settings ACCEPT
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience. You can read our privacy policy for more information.
Cookie Consent
Save & Accept