How to Add Sitemap to Robots.txt and Have Your Site Indexed by Search Engines
Site maps are essential to get all permalinks of your site indexed by search engines. Using a standard Sitempas protocol, Ask.com, Google, Microsoft Live Search and Yahoo! have announced support of “autodiscovery” of Sitemaps in your website root directory. The new open-format autodiscovery allows webmasters to specify the location of their Sitemaps within their robots.txt file, eliminating the need to submit sitemaps to each search engine separately.
This step is very useful as webmasters can easily submit their content to the search engines and benefit from reduced unnecessary traffic by the crawlers. The search engines get information with regards to pages so index as well as metadata with clues about which pages are newly updated and which pages are identified as the most important and search users get more fresh content.
How to modify the robots.txt file?
Simply add the sitemap url following the word sitemap like this
Sitemap: http://www.internet-marketing.web.id/sitemap.xml
(Replace internet-marketing.web.id with your own site name.)
Save & upload the file and you’re done! Its simple. Update your robots.txt file today and help search engines index your site better.
