Improve your Search Engine Position with Sitemaps

The term "sitemap" can refer to two different things.  Many large, complex Web sites provide a visual sitemap that visitors can use for quick navigation, if they already know roughly where they want to go.  If your site is large or complex, you should provide one of these sitemaps for your visitors.

But this article is about the other kind of sitemap:  The kind that is made for the search engines, like Google, to use in indexing your site.  There are several forms that these sitemaps can take, but we'll get to that a little later.

First of all, let's consider why you even need a sitemap.  Google and the other search engines will index your site even if you don't have a sitemap.  However, there are four main advantages to having a sitemap:

1. If your site uses non-HTML links, such as Macromedia Flash menus or JavaScript menus, the search engines will not be able to follow these links, and so they will not find all of your pages.  A code-driven site must use a sitemap.
2. A sitemap tells the search engines which pages on your site are more important, and which are less important.  This prevents the less important pages from competing with your own pages in the listings.
3. A sitemap tells the search engines which pages on your site are updated more frequently than others.  This enables the search engines to ignore your static pages, increasing the likelihood that they will have the most current data on your most dynamic pages.
4. A sitemap enables you to tell the search engines when you have added or updated your site's content.  To some extent, this puts you in control of making the search engines aware of your latest content.  Of course, it doesn't force the search engines to do your bidding, but it tends to make it easier for users to find your new pages more quickly.

So, what is a sitemap?

As mentioned above, there are many possible forms of sitemaps, but we'll concentrate on the most useful kind, the XML sitemap format created and promulgated by sitemaps.org.  This protocol, currently known as "Sitemap 0.90," is maintained and endorsed jointly by Google, MSN, Yahoo, and Ask, so you know it is pretty much a universal standard.

An XML sitemap consists of a list of pages on your Web site, and standard information about each page.  Here is an example:

   http://www.freelancesubmit.com/Index.htm  2008-04-07  never  0.3 

...   http://www.freelancesubmit.com/Services.htm  2008-04-07  weekly  0.8 

...

Don't worry about the technical details of formatting the XML.  We'll talk about tools that will create this for you in a moment.

There are three things to notice about each entry:

1. LastMod.  Tell the search engines the last date (and time) you changed this page.  That will tell them which ones they ought to index right away, and which ones they can ignore.
2. ChangeFreq.  In case you're not updating your sitemap all the time, this will give the search engines a clue as to how often they ought to check each page.
3. >Priority.  This tells the search engines the relative importance of this page, compared to all the other pages in your site.

In assigning a value for "Priority," on a scale of 0.0 to 1.0, determine which pages are most important and which are least important within your site.  We're not telling the search engines that this "Services" page is in the 80th percentile of all pages on the Web, but it is far more important than the "Index" page within this site.  That's where we want our visitors to end up.

It's easy to identify pages within your site which are lowest priority.  Some examples:

• Privacy Policy
• "Contact us"
• "About us"

Please don't misunderstand this.  It's not that your "Privacy Policy" page is unimportant and so you might as well not have one.  It's that your "Privacy Policy" is important enough to take for granted:  Your visitors will find it when they need it.  But for search engine purposes, you'd rather direct them to the pages where you actually do your business.

So, how do you create a sitemap?

There are a number of software tools that will create a sitemap by reading your site's content.  You will have to adjust the results, especially the "Priority" settings, but most of these do a pretty good job.  Search the Web for "sitemap generator," or for any of the following specific free tools:

• SitemapDoc
• XML-Sitemaps
• AuditMyPC Google Sitemap Generator

And once you have your sitemap, what do you do with it?

There are three things to do, in sequence:

1. Place the sitemap file into the root directory of your Web server, alongside your main "index" file.  And each time you update it, place the new copy there.
2. Notify the major search engines of your new sitemap file each time you update it.  For Google, this means to submit it from within "Webmaster Tools."  For other major search engines, search on that search engine for "submit sitemap," and you'll probably find where to enter the URL of your sitemap file.
3. Place a reference to the sitemap file in your robots.txt file, as "Sitemap: http://www.freelancesubmit.com/sitemap.xml".  This will make sure that any search engine will find it, even those that you did not submit it to directly.  You only need to do this once, unless you change the name or location of your sitemap file.

Once you have your sitemap created and submitted, don't forget to maintain it.  Each time you add a page to your Web site, add it to your sitemap.  Each time you update a page on your Web site, update its "lastmod" setting in your sitemap.  Try adjusting the "priority" of your pages from time to time to see if it improves the performance of that particular page.  And each time you modify your sitemap, resubmit it to the major search engines.