Hi,
I put the "Options All -Indexes" on my subdirectories to block access to the listing of files in those directories. It works, but along with the "Forbidden" page, it also adds there is an additional error 404, page not found. This is because the Linux OS with Apache server is looking for the index.html page, the default page. Here's the problem, if I write a blank index.html, like this:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1" http-equiv="content-type">
<title>Forbidden</title>
<meta name="robots" content="noindex">
</head>
<body>
</body>
</html>
Then instead of giving me the Forbidden page, it just shows the blank index.html page. Aaaaarg!
What I wanted was the forbidden page to show up WITHOUT the 404 message. Anyway to get that and also, in regard to SEO, what do bots do.. do they only find pages by following links or do the search your directories for pages, even though they are not linked? (In case they do, that's the reason I added the "noindex " to the blank index.html file.
Thanks,
Pete
Subscribe to HRA Now!
Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!
www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
International SEM | Social Media | Search Friendly Design | SEO | Paid Search / PPC | Seminars | Forum Threads | Q&A | Copywriting | Keyword Research | Web Analytics / Conversions | Blogging | Dynamic Sites | Linking | SEO Services | Site Architecture | Search Engine Spam | Wrap-ups | Business Issues | HRA Questions | Online Courses
Sponsored Content
Problems With .htaccess And Options All -indexes
Started by
Pete L
, Jun 29 2009 01:49 PM
3 replies to this topic
#2
Randy
-
- Moderator
-
- 17,540 posts
Convert Me!
Posted 29 June 2009 - 02:17 PM
I think I may know what's happening Pete, but it's a little convoluted. Like most things with servers seems to be convoluted. 
The -Indexes by default will send a 403 Forbidden status code. Which is also the page Apache is going to attempt to send those hits to.
My guess, and it's pure speculation based upon things I've seen in the past, is that you don't have a 403 page specified anywhere in your domain configuration. Thus when Apache tries to first deliver the 403 Forbidden status message, then tries to display your 403 page it bumps up against there being no defined file and thus does the next best thing and displays a 404 error for your 403 page.
See, I told ya it was convoluted.
Try this as a little test. Upload a test file to your server somewhere, the root level will probably work best in most situations. Name it something nice and logical like 403.html, or if you want to use server side scripting on it make it 403.php or whatever.
Then in your .htaccess define this new page like:
Then try to directly access one of your directories that don't have a index page again. If my guess is right you should now get your test page, no matter which directory index listing you try to access. If the test works you can now pretty it up a bit and customize your new 403 page.
re: The Search Engines... Don't worry about them much for this type of thing. As long as the server is delivering a solid 4xx series status code they'll leave those pages alone.
The -Indexes by default will send a 403 Forbidden status code. Which is also the page Apache is going to attempt to send those hits to.
My guess, and it's pure speculation based upon things I've seen in the past, is that you don't have a 403 page specified anywhere in your domain configuration. Thus when Apache tries to first deliver the 403 Forbidden status message, then tries to display your 403 page it bumps up against there being no defined file and thus does the next best thing and displays a 404 error for your 403 page.
See, I told ya it was convoluted.
Try this as a little test. Upload a test file to your server somewhere, the root level will probably work best in most situations. Name it something nice and logical like 403.html, or if you want to use server side scripting on it make it 403.php or whatever.
Then in your .htaccess define this new page like:
CODE
ErrorDocument 403 /403.html
Then try to directly access one of your directories that don't have a index page again. If my guess is right you should now get your test page, no matter which directory index listing you try to access. If the test works you can now pretty it up a bit and customize your new 403 page.
re: The Search Engines... Don't worry about them much for this type of thing. As long as the server is delivering a solid 4xx series status code they'll leave those pages alone.
#3
Pete L
-
- Active Members
-
- 103 posts
HR 3
Posted 29 June 2009 - 08:32 PM
Try this as a little test. Upload a test file to your server somewhere, the root level will probably work best in most situations. Name it something nice and logical like 403.html, or if you want to use server side scripting on it make it 403.php or whatever.
Then in your .htaccess define this new page like:
Then try to directly access one of your directories that don't have a index page again. If my guess is right you should now get your test page, no matter which directory index listing you try to access. If the test works you can now pretty it up a bit and customize your new 403 page.
OK, thanks a lot, that worked!
It is interesting that another way around the 404 error that doesn't involve having a 403.html page is to just code:
It seems the server will just return the message set after the 403 to the screen. I decided to go with a 403.html page, anyway telling the visitor they don't have permission to access this area of the server... just in case they haven't seen Home Improvement. And speaking of which, hey Randy, where is Jill these days?
Pete
Then in your .htaccess define this new page like:
CODE
ErrorDocument 403 /403.html
Then try to directly access one of your directories that don't have a index page again. If my guess is right you should now get your test page, no matter which directory index listing you try to access. If the test works you can now pretty it up a bit and customize your new 403 page.
OK, thanks a lot, that worked!
It is interesting that another way around the 404 error that doesn't involve having a 403.html page is to just code:
CODE
ErrorDocument 403 I don't think so, Tim!
It seems the server will just return the message set after the 403 to the screen. I decided to go with a 403.html page, anyway telling the visitor they don't have permission to access this area of the server... just in case they haven't seen Home Improvement. And speaking of which, hey Randy, where is Jill these days?
Pete
#4
Randy
-
- Moderator
-
- 17,540 posts
Convert Me!
Posted 29 June 2009 - 09:04 PM
She's here somewhere. I saw her wandering around the forums early this morning.
Other than that I have no idea what she's actually up to. Been too busy with a couple of projects myself to even turn on my MSN Messenger or Twitter for the last week or two. You might want to check out her Twitter account since she posts stuff to it pretty much daily. I'm not nearly that dedicated to tweeting apparently.
Other than that I have no idea what she's actually up to. Been too busy with a couple of projects myself to even turn on my MSN Messenger or Twitter for the last week or two. You might want to check out her Twitter account since she posts stuff to it pretty much daily. I'm not nearly that dedicated to tweeting apparently.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users
