Yet another reason DNN should be cast into space
- anyone know of a method by which we can block crawling of a couple of these subdomains?
Subscribe to HRA Now!
Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!
www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
International SEM | Social Media | Search Friendly Design | SEO | Paid Search / PPC | Seminars | Forum Threads | Q&A | Copywriting | Keyword Research | Web Analytics / Conversions | Blogging | Dynamic Sites | Linking | SEO Services | Site Architecture | Search Engine Spam | Wrap-ups | Business Issues | HRA Questions | Online Courses
Sponsored Content
Complex Issue Blocking Subdomains...
Started by
grooveitgolf_com
, Sep 18 2007 10:29 AM
4 replies to this topic
#1
grooveitgolf_com
-
- Active Members
-
- 57 posts
HR 3
- Location:Charleston, SC
Posted 18 September 2007 - 10:29 AM
One of our client sites is built in Dotnetnuke, and their subdomains are all portals in the root directory. Hence, we can't stick a robots.txt in the root of each folder.
Yet another reason DNN should be cast into space - anyone know of a method by which we can block crawling of a couple of these subdomains?
Yet another reason DNN should be cast into space
- anyone know of a method by which we can block crawling of a couple of these subdomains?
#2
Randy
-
- Moderator
-
- 17,540 posts
Convert Me!
Posted 18 September 2007 - 11:20 AM
I'm not DNN person by choice 
but this post by Chris should help you in setting up a scripted robots.txt for your subdomains.
but this post by Chris should help you in setting up a scripted robots.txt for your subdomains.
#3
Ron Carnell
-
- Moderator
-
- 959 posts
HR 6
- Location:Michigan USA
Posted 18 September 2007 - 12:37 PM
I'm not a DNN person either . . . but I question whether that's really an issue.
When you say "their subdomains are all portals in the root directory," I take that to mean that each subdomain resides in a folder within the root; right? Okay, fine. That doesn't explain why you can't stick a robots.txt file in the root of each folder?
On the contrary, that is exactly what you should do, in my opinion.
Spiders won't look for a robots.txt outside the document root, so the robots.txt files in the sub-folders won't be found by a spider visiting the primary domain. However, those sub-folders ARE the document root for the corresponding subdomain, so that's where a spider visiting each subdomain will look for them. Make sense?
When you say "their subdomains are all portals in the root directory," I take that to mean that each subdomain resides in a folder within the root; right? Okay, fine. That doesn't explain why you can't stick a robots.txt file in the root of each folder?
On the contrary, that is exactly what you should do, in my opinion.
Spiders won't look for a robots.txt outside the document root, so the robots.txt files in the sub-folders won't be found by a spider visiting the primary domain. However, those sub-folders ARE the document root for the corresponding subdomain, so that's where a spider visiting each subdomain will look for them. Make sense?
#4
chrishirst
-
- Moderator
-
- 5,880 posts
A not so moderate moderator.
- Location:Blackpool UK
Posted 18 September 2007 - 01:42 PM
QUOTE
Yet another reason DNN should be cast into space
I would love to be there for that launch date!!!
#5
grooveitgolf_com
-
- Active Members
-
- 57 posts
HR 3
- Location:Charleston, SC
Posted 19 September 2007 - 03:32 PM
QUOTE(Ron Carnell @ Sep 18 2007, 01:37 PM) 
I'm not a DNN person either . . . but I question whether that's really an issue.
When you say "their subdomains are all portals in the root directory," I take that to mean that each subdomain resides in a folder within the root; right? Okay, fine. That doesn't explain why you can't stick a robots.txt file in the root of each folder?
On the contrary, that is exactly what you should do, in my opinion.
Spiders won't look for a robots.txt outside the document root, so the robots.txt files in the sub-folders won't be found by a spider visiting the primary domain. However, those sub-folders ARE the document root for the corresponding subdomain, so that's where a spider visiting each subdomain will look for them. Make sense?
When you say "their subdomains are all portals in the root directory," I take that to mean that each subdomain resides in a folder within the root; right? Okay, fine. That doesn't explain why you can't stick a robots.txt file in the root of each folder?
On the contrary, that is exactly what you should do, in my opinion.
Spiders won't look for a robots.txt outside the document root, so the robots.txt files in the sub-folders won't be found by a spider visiting the primary domain. However, those sub-folders ARE the document root for the corresponding subdomain, so that's where a spider visiting each subdomain will look for them. Make sense?
Sort've. The subdomains and their pages appear to be generated dynamically. There are folders there in the root, but they have names like "0" and "1" instead of subdomains. Each folder seems to apply to a skin, rather than a subdomain so there really is no place to put those other robots files. I could be wrong, so if there's anyone else out there that is stupid enough to get tangled up in DNN, give me a shout.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users
