Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!

www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
International SEM | Social Media | Search Friendly Design | SEO | Paid Search / PPC | Seminars | Forum Threads | Q&A | Copywriting | Keyword Research | Web Analytics / Conversions | Blogging | Dynamic Sites | Linking | SEO Services | Site Architecture | Search Engine Spam | Wrap-ups | Business Issues | HRA Questions | Online Courses
Robots.txt & Sitemap.xml Redundant?
Started by
Tyl3r
, Aug 15 2007 03:09 PM
7 replies to this topic
#1
Posted 15 August 2007 - 03:09 PM
Hello all and thanks in advance. In your experience is having Robots.txt & Sitemap.xml for the same website redundant? Thanks!
#2
Posted 15 August 2007 - 05:22 PM
well, as they serve two totally different purposes, why would you think that??
#3
Posted 15 August 2007 - 05:50 PM
You can use robots.txt to point to the sitemap, so they can work together, but of course you need both to do that.
#4
Posted 15 August 2007 - 06:17 PM
well, as they serve two totally different purposes, why would you think that??
Oh, OK, if both have different purposes which would you say is more crucial to have? And don't they both function as ways for the robots to see all of the files available for spidering? If so, then what exactly is the difference? Thanks for your help on this.
#5
Posted 15 August 2007 - 06:21 PM
Nope,
robots.txt is for EXCLUDING urls from compliant crawlers.
and neither are crucial or even necessary.
robots.txt is for EXCLUDING urls from compliant crawlers.
and neither are crucial or even necessary.
#6
Posted 15 August 2007 - 06:28 PM
So a robots.txt basically tells the robots what to do with a series of files....OK, that makes sense. Thanks again, you rule Comic Book Store Owner dude.
#7
Posted 16 August 2007 - 03:26 AM
Not quite.
robots.txt tells a bot what NOT to do with files or folders
robots.txt tells a bot what NOT to do with files or folders
#8
Posted 17 August 2007 - 02:15 AM
Why is this an either/or situation?
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users








