SEO Class in Chicago, IL
Learn How To Optimize Your Website on July 26, 2013
High Rankings is offering a 1-day customized SEO training class in Chicago. Class size is limited so please sign-up now if you want in!
Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!

www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
Robots.txt & Sitemap.xml Redundant?
#1
Posted 15 August 2007 - 03:09 PM
#2
Posted 15 August 2007 - 05:22 PM
#3
Posted 15 August 2007 - 05:50 PM
#4
Posted 15 August 2007 - 06:17 PM
Oh, OK, if both have different purposes which would you say is more crucial to have? And don't they both function as ways for the robots to see all of the files available for spidering? If so, then what exactly is the difference? Thanks for your help on this.
#5
Posted 15 August 2007 - 06:21 PM
robots.txt is for EXCLUDING urls from compliant crawlers.
and neither are crucial or even necessary.
#6
Posted 15 August 2007 - 06:28 PM
#7
Posted 16 August 2007 - 03:26 AM
robots.txt tells a bot what NOT to do with files or folders
#8
Posted 17 August 2007 - 02:15 AM
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users









