is there a difference between:
User-agent: *
Disallow: /searchprod/
and
User-agent: *
Disallow: /searchprod
...basically removing the last forward slash?
Subscribe to HRA Now!
SEO Class in Chicago, IL
Learn How To Optimize Your Website on July 26, 2013
Looking for personalized in-depth SEO training among your peers?
High Rankings is offering a 1-day customized SEO training class in Chicago. Class size is limited so please sign-up now if you want in!
Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!
www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
International SEM | Social Media | Search Friendly Design | SEO | Paid Search / PPC | Seminars | Forum Threads | Q&A | Copywriting | Keyword Research | Web Analytics / Conversions | Blogging | Dynamic Sites | Linking | SEO Services | Site Architecture | Search Engine Spam | Wrap-ups | Business Issues | HRA Questions | Online Courses
Quick Little Robots.txt Question
Started by
thwart
, Oct 15 2007 10:27 AM
3 replies to this topic
#2
Alan Perkins
-
- Admin
-
- 1,566 posts
Token male admin
- Location:UK
Posted 15 October 2007 - 10:31 AM
Yes.
User-agent: *
Disallow: /searchprod/
will disallow everything in the searchprod directory.
User-agent: *
Disallow: /searchprod
will disallow everything in the searchprod directory, and everything in the root directory that begins with searchprod, e.g. /searchproducts.html
This is is because disallow lines in robots.txt are a pattern match to the start of a URL.
User-agent: *
Disallow: /searchprod/
will disallow everything in the searchprod directory.
User-agent: *
Disallow: /searchprod
will disallow everything in the searchprod directory, and everything in the root directory that begins with searchprod, e.g. /searchproducts.html
This is is because disallow lines in robots.txt are a pattern match to the start of a URL.
#3
thwart
-
- Active Members
-
- 59 posts
HR 3
Posted 15 October 2007 - 10:38 AM
QUOTE(Alan Perkins @ Oct 15 2007, 11:31 AM) 
Yes.
User-agent: *
Disallow: /searchprod/
will disallow everything in the searchprod directory.
User-agent: *
Disallow: /searchprod
will disallow everything in the searchprod directory, and everything in the root directory that begins with searchprod, e.g. /searchproducts.html
This is is because disallow lines in robots.txt are a pattern match to the start of a URL.
User-agent: *
Disallow: /searchprod/
will disallow everything in the searchprod directory.
User-agent: *
Disallow: /searchprod
will disallow everything in the searchprod directory, and everything in the root directory that begins with searchprod, e.g. /searchproducts.html
This is is because disallow lines in robots.txt are a pattern match to the start of a URL.
Ah, so if I don't have any files that have searchprod in them, THEN the two different syntaxes would be pretty much doing the same thing. Interesting.
#4
Alan Perkins
-
- Admin
-
- 1,566 posts
Token male admin
- Location:UK
Posted 15 October 2007 - 10:44 AM
QUOTE(thwart)
if I don't have any files that have searchprod in them, THEN the two different syntaxes would be pretty much doing the same thing.
Correct.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users
