Manually submitting XML site maps to search engines is now outdated thanks to an internet-wide agreement reached by search engines.
From now on, adding an extra line to your robots.txt file will tell search engines exactly where to go to find your search map rather than leaving it to chance that they find it for themselves. The line you need to add is: Sitemap: http://www.yourdomain.com/sitemap.xml
Along with last autumn’s agreement on a standard format for site maps, promoting your web site has been made a good deal more efficient. Now, the weak link is the sitemap’s accuracy. Unless the site map is kept up to date, the new developments will be of no benefit.
Most small to micro-sized business sites get by with manual XML site map creation, but if you carry out a lot of updates, frequently add or delete pages or have a very large site then an automated system is more efficient and less time-consuming. Automation is usually in the form of script that runs directly on the web site’s server, and creates and uploads a new version of the site map after each update.
The new robots.txt file standard makes for a more pro-active approach to web site promotion. Until now robots.txt files were as much about telling search engine spiders where they weren’t allowed to go than where they were permitted to crawl.
