The Importance of Sitemaps
The Importance of Sitemaps
Thanks to the Sitemap protocol, the webmaster can inform search engines what URLs from the site are available for crawling.
This is possible by creating a special XML file that lists the available URLs along with additional and useful for the search engines information - how often and when they're updated, and what their relative importance is.
This information allows search engines to gain a clearer picture of the site structure and as a result to index the content objectively and intelligently.
robots.txt protocol that specifies which parts of the site should not get indexed, the Sitemap protocol does exactly the opposite - recommends which URLs on the site should be indexed.
Both tools are not contradictory but complement each other providing a clear picture of the site to search engines.
Sitemap and page rank
A Sitemap does not affect the actual ranking of web pages. It notifies SEs about new pages they did not know or crawled yet and it helps them to better prioritize the URLs of the site. As a result that can lead to increased presence and visibility of the site in SE index.
HTML and XML Sitemap
There's a difference between a (usually HTML) site map built to help visitors navigate around web site, and an XML Sitemap built for search engines. Both of them are useful, and it's great to have both.
The advantage of XML Sitemap is the additional information it gives to search engines, while HTML Sitemap is just a collection of links.
General Sitemap Guidelines
- Use consistent, fully-qualified URLs. Search engines will crawl your URLs exactly as listed. For instance, if your site is at
http://www.example.com/, don't specify a URL as
http://example.com/(without the www) or
./mypage.html(a relative URL).
- Place your Sitemap at the root directory of your HTML server -
- Sitemap files must be UTF-8 encoded.
- Break up large sitemaps into a smaller sitemaps to prevent your server from being overloaded. A sitemap file can't contain more than 50,000 URLs and must be no larger than 50 MB uncompressed.
- Use a sitemap index file to list all your sitemaps and submit this single file rather than submitting individual sitemaps.
- Use recommended canonicalization methods to tell search engines if your site is accessible on both the www and non-www versions of your domain. You need to submit a sitemap for only your preferred domain.
Build a Sitemap
If you’re not using WordPress and your CMS doesn’t auto-generate sitemaps, we can recommend a free tool that can create a sitemap for you.
Creating a sitemap XML file with XML-Sitemaps
First, go to XML-Sitemaps and paste in your website URL:
Click on Start button to start the crawling process.
If your website has over 500 URLs, you’ll need to setup a paid account in order to continue using the tool.
Download the sitemap (the default format is .xml) and upload it through FTP in the root folder of your website:
Submit a Sitemap
Reference your sitemap in robots.txt
You can also leave a reference to your sitemap in your site’s robot.txt file. Insert the following line to your robots.txt, replacing the example sitemap location with the location of your sitemap or sitemap index:
Submit Sitemaps to Google
Select your site on your Google Search Console home page.
- Click Crawl
- Click Sitemaps
- Click ADD/TEST SITEMAP
- Type sitemap.xml
- Click Submit Sitemap
Test your sitemap using the Search Console Sitemaps testing tool.
Submit Sitemaps to Bing
- Submit sitemaps from your Dashboard
- Submit sitemaps from the Sitemaps Feature
Pinging Bing Using a HTTP request
Use the following URL, replacing the example with your sitemap location and URL encoding everything behind the sitemap parameter:
The server will respond with HTTP Status 200 (OK) if the submission was successful. Unlike the submission options inside Webmaster Tools you will not get any information about the validity about the actual sitemap itself.
The Sitemap Protocol format consists of XML tags. The file itself must be UTF-8 encoded. Sitemaps can also be just a plain text list of URLs. They can also be compressed in
A sample Sitemap that contains just one URL and uses all optional tags is shown below.
<?xml version="1.0" encoding="utf-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"> <url> <loc>http://example.com/</loc> <lastmod>2006-11-18</lastmod> <changefreq>daily</changefreq> <priority>0.8</priority> </url> </urlset>
The Sitemap XML protocol is also extended to provide a way of listing multiple Sitemaps in a 'Sitemap index' file.
An example of Sitemap index referencing one separate sitemap follows.
<?xml version="1.0" encoding="UTF-8"?> <sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>http://www.example.com/sitemap1.xml.gz</loc> <lastmod>2014-10-01T18:23:17+00:00</lastmod> </sitemap> </sitemapindex>
The following XML schemas define the elements and attributes that can appear in your Sitemap file. You can download this schema from the links below:
For Sitemap index files:
In order to validate your Sitemap or Sitemap index file against a schema, the XML file will need additional headers as shown below.
<?xml version='1.0' encoding='UTF-8'?> <urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> ... </url> </urlset>
Sitemap index file
<?xml version='1.0' encoding='UTF-8'?> <sitemapindex xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> ... </sitemap> </sitemapindex>
Sitemaps can offer the opportunity to link search engines with any changes made to the site immediately after they happen.
A well-structured sitemap will make a website searchable by all search engines and will provide users with more accurate search results.
#apps #tools #seo