How to Use Nofollow, Noindex and Robots.txt for SEO

Posted on Nov 11, 2008 by Rick to SEO

There’s often confusion about how nofollow, noindex and robots.txt should be used for good SEO. I’m going to give an overview of the three, including when, where and why you should be using these.

What is a robots.txt file
First, a robots.txt file is simply a file that you can place on your web server which will give instructions to search engine robots with regards to which sections and pages they should not crawl. This is the equivalent of putting a ‘do not disturb’ sign on your hotel room door. While most will obey your request, there is no guarantee. The reasons for blocking robots from visiting site pages and sections vary by website but are usually related to not wanting the search engine to crawl or index select pages or site sections.

What does the noindex meta tag do?
The noindex Meta tag is a simple tag that instructs the search engine not to display a given page in the search index (search results). This tag is often used when a website owner does not want select pages appearing in search. Websites with duplicate content issues often use these tag to remove the duplicate content from the search index.

Placing a noindex tag on a page is different than excluding that page in robots.txt file in that a page with noindex in the header can still be crawled by the search engines and can still accumulate and pass PageRank. A noindex tag will simply put any given page in invisible mode to searchers.

What does the nofollow tag do?
Nofollow tags can be inserted in the header or attached to links to provide instructions for the search engines like Google not to crawl a given link. In essence, this prevents PageRank from being passed from one page to another when nofollow is used. This is done for a couple reasons:

1. Spam reduction – Most websites allow users to contribute in some way. By placing nofollow links on any links added by users, you can discourage users from blatantly spamming your website because the links will carry no SEO value.
2. PageRank sculpting – For webmasters concerned with the distribution of PageRank throughout their website, nofollow can be placed on links pointing to unimportant pages within their website. For example, a link going to your privacy policy page could use nofollow because it isn’t likely that this page is very important to the search engines. Adding nofollow to all links pointing at this page will mean that more PageRank is passed to the other more important pages on your website.

Utilizing robots.txt, nofollow and noindex can make a noticeable impact on your organic search marketing results if used correctly. However, using them incorrectly can lead to adverse effects such as ranking pages being removed from the indexed and loss of PageRank. Apply them with caution.

Tags: nofollow, noindex, pagerank, robots.txt

PageRank Sculpting- Maximizing the Impact of the Links You’ve Earned

Posted on Oct 15, 2008 by Rick to SEO

When it comes to SEO, link building is probably the most time-consuming part of most campaigns. This is why it’s important to tweak every onsite factor possible to maximize the impact of the inbound links that you’ve acquired. One tactic we’ve used in the past here at THAT Agency is a tactic known as PageRank sculpting. This tactic is geared towards maximizing the impact of the inbound links that a website already has on search engine rankings.

What is PageRank Sculpting?
PageRank sculpting is a technique where, by adjusting onsite elements like website architecture and applying nofollow tags to chosen links, you can guide the search engines in assigning PageRank to the most important pages of your website. Ideally, you’ll want to guide PageRank to pages that you’ve optimized for search and that will convert well with your visitors.

In general, most websites have the strongest PageRank on the homepage. This is because PageRank is determined mostly by inbound links and most people will link to a website’s homepage more often than to an interior landing page. Since PageRank can be passed from one page to another when one page links to another page, how you design your website navigation will affect the way that Google assigns PageRank to internal pages of your website.

For example, internal pages linked to from the homepage will look, in the eyes of Google, as important pages because the most important page on the website, the homepage, links to them. Pages not linked to from the homepage but linked to from other pages on the site will still get PageRank, but not as much. Also, on most websites, the navigation area and footer appears on each and every page of the website. Usually, these areas content links. In the eyes of Google, any page that is linked to from many internal pages also appears to be important and thus, will likely receive more PageRank than internal pages with fewer internal links. These are only two examples of how your website architecture will affect your PageRank. The important thing to remember is that your most important pages should be linked to from your homepage and from the global navigation, if possible.

The NoFollow Attribute
On many sites, the global navigation links to the ‘About us’ or ‘Privacy Policy’ pages. From the user experience perspective, this is necessary. From a search engine perspective, this can be a bad thing. By linking to these types of pages, which aren’t typically optimized for search, you’re passing PageRank that could better be used on pages that are optimized for search.

This is where the nofollow attribute comes into play. It looks like this html:

a href=”http://www.yourwebsite.com/page.php” rel=”nofollow”

The nofollow attribute can be applied to any link and tells Google not to pass PageRank to the page on the other end. In effect, this means that a little more PageRank will be passed to each of the other links on the same page. By applying this attribute to global navigation links which link to non-important pages (form the perspective of search engines), you can force PageRank to be passed to the more important pages.

The three important things to remember about PageRank Sculpting are:
* Link to the important pages on your website from your homepage
* Link to the important pages on your website often
* If you have unimportant links (privacy policy, about us, etc.) in your global navigation, apply the nofollow attribute

Tags: pagerank

Adwords Will Increase Your Page Rank (PR)

Posted on Dec 8, 2006 by Bill to SEO

During a HSMAI travel internet marketing strategy conference in Sunny Isles Florida yesterday we were fortunate enough to have a panel that comprised three top level executives from Google. As would be expected a member asked the panel “what can they do to have their site indexed quicker”.

Google’s team went through the channels that most of us are familiar with explaining that the site must be built properly, setup Google’s sitemap and obtain natural listings. The members then complained that they had done these eight months ago and still have a page rank of 0. Google’s team then explained that Google is not the one that decides how quickly your site is ranked that the general public decides by how often your site is visited. Google then explained that using Google’s Adwords is a great way to drive traffic to the site, hence creating traffic. This is only one aspect of the algorithms that the search engines use but today you may need to buy your way to the position you need.

Tags: pagerank