Home > Help Center > Shared Hosting > Website > The Importance of Sitemaps

The Importance of Sitemaps

The Importance of Sitemaps

Thanks to the Sitemap protocol, the webmaster can inform search engines what URLs from the site are available for crawling.

This is possible by creating a special XML file that lists the available URLs along with additional and useful for the search engines information - how often and when they're updated, and what their relative importance is.

This information allows search engines to gain a clearer picture of the site structure and as a result to index the content objectively and intelligently.

Unlike the robots.txt protocol that specifies which parts of the site should not get indexed, the Sitemap protocol does exactly the opposite - recommends which URLs on the site should be indexed.

Both tools are not contradictory but complement each other providing a clear picture of the site to search engines.

Sitemap and page rank

A Sitemap does not affect the actual ranking of web pages. It notifies SEs about new pages they did not know or crawled yet and it helps them to better prioritize the URLs of the site. As a result that can lead to increased presence and visibility of the site in SE index.

HTML and XML Sitemap

There's a difference between a (usually HTML) site map built to help visitors navigate around web site, and an XML Sitemap built for search engines. Both of them are useful, and it's great to have both.

The advantage of XML Sitemap is the additional information it gives to search engines, while HTML Sitemap is just a collection of links.

General Sitemap Guidelines

  • Use consistent, fully-qualified URLs. Search engines will crawl your URLs exactly as listed. For instance, if your site is at http://www.example.com/, don't specify a URL as http://example.com/ (without the www) or ./mypage.html (a relative URL).
  • Place your Sitemap at the root directory of your HTML server - http://example.com/sitemap.xml
  • Sitemap files must be UTF-8 encoded.
  • Break up large sitemaps into a smaller sitemaps to prevent your server from being overloaded. A sitemap file can't contain more than 50,000 URLs and must be no larger than 50 MB uncompressed.
  • Use a sitemap index file to list all your sitemaps and submit this single file rather than submitting individual sitemaps.
  • Use recommended canonicalization methods to tell search engines if your site is accessible on both the www and non-www versions of your domain. You need to submit a sitemap for only your preferred domain.

Build a Sitemap

If you’re not using WordPress and your CMS doesn’t auto-generate sitemaps, we can recommend a free tool that can create a sitemap for you.

Creating a sitemap XML file with XML-Sitemaps

First, go to XML-Sitemaps and paste in your website URL:

sitemap

Click on Start button to start the crawling process.

If your website has over 500 URLs, you’ll need to setup a paid account in order to continue using the tool.

Download the sitemap (the default format is .xml) and upload it through FTP in the root folder of your website:

sitemap

Submit a Sitemap

Reference your sitemap in robots.txt

You can also leave a reference to your sitemap in your site’s robot.txt file. Insert the following line to your robots.txt, replacing the example sitemap location with the location of your sitemap or sitemap index:

Sitemap: http://www.example.com/sitemap.xml

Submit Sitemaps to Google

Select your site on your Google Search Console home page.

  • Click Crawl
  • Click Sitemaps
  • Click ADD/TEST SITEMAP
  • Type sitemap.xml
  • Click Submit Sitemap

Test your sitemap using the Search Console Sitemaps testing tool.

Submit Sitemaps to Bing

  • Submit sitemaps from your Dashboard
  • Submit sitemaps from the Sitemaps Feature

Pinging Bing Using a HTTP request

Use the following URL, replacing the example with your sitemap location and URL encoding everything behind the sitemap parameter:

http://www.bing.com/ping?sitemap=http%3A%2F%2Fwww.example.com/sitemap.xml

The server will respond with HTTP Status 200 (OK) if the submission was successful. Unlike the submission options inside Webmaster Tools you will not get any information about the validity about the actual sitemap itself.

Sitemap Format

The Sitemap Protocol format consists of XML tags. The file itself must be UTF-8 encoded. Sitemaps can also be just a plain text list of URLs. They can also be compressed in .gz format.

A sample Sitemap that contains just one URL and uses all optional tags is shown below.

<?xml version="1.0" encoding="utf-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" 
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
    <url>
        <loc>http://example.com/</loc>
        <lastmod>2006-11-18</lastmod>
        <changefreq>daily</changefreq>
        <priority>0.8</priority>
    </url>
</urlset>

The Sitemap XML protocol is also extended to provide a way of listing multiple Sitemaps in a 'Sitemap index' file.

An example of Sitemap index referencing one separate sitemap follows.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.example.com/sitemap1.xml.gz</loc>
      <lastmod>2014-10-01T18:23:17+00:00</lastmod>
   </sitemap>
</sitemapindex>

Sitemap Validation

The following XML schemas define the elements and attributes that can appear in your Sitemap file. You can download this schema from the links below:

For Sitemaps: http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd For Sitemap index files: http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd

In order to validate your Sitemap or Sitemap index file against a schema, the XML file will need additional headers as shown below.

Sitemap

<?xml version='1.0' encoding='UTF-8'?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
         xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      ...
   </url>
</urlset>

Sitemap index file

<?xml version='1.0' encoding='UTF-8'?>
<sitemapindex xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd"
         xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      ...
   </sitemap>
</sitemapindex>

Conclusion

Sitemaps can offer the opportunity to link search engines with any changes made to the site immediately after they happen.

A well-structured sitemap will make a website searchable by all search engines and will provide users with more accurate search results.

#apps #tools #seo

Still not finding what you're looking for?

Contact our support team with any additional questions or concerns.

Contact support