Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!


Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 




From the folks who brought you High Rankings!

- - - - -

Was I Sandboxed?

  • Please log in to reply
16 replies to this topic

#16 andruha


    HR 3

  • Active Members
  • PipPipPip
  • 60 posts

Posted 23 March 2010 - 10:55 PM

QUOTE(Randy @ Mar 23 2010, 02:18 PM) View Post
First thing I'd do is double check to make sure some entry in the index page's <meta robots= ... > section or your robots.txt isn't telling the spiders to stay away from that page.

Second thing I'd do is review any .htaccess or other redirect instructions to make sure one isn't trying to redirect from www.domain.com/index.ext to www.domain.com and is doing so in such a way that it's creating a loop, causing the page to never actually load for spiders. Most browsers will dump out of such endless loops eventually and will usually end up showing the page. Spiders don't. They'll dump out eventually, but will never get the page in question.

Thanks, never thought of checking those files. But wouldn't it block the page for other spiders, such as bing and yahoo? I'm still on the first page of Bing.

My meta tag is
<meta name="robots" content="follow, all" />

and my robots.txt is
User-agent: *

Sitemap: http://XXXXXX.info/sitemap.xml.gz

I ran a test on GW and got this result (what does it mean?):
Allowed by line 2: Disallow:
Detected as a directory; specific files may have different restrictions

and here is my .thaccess:
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
# END WordPress
<Files 403.shtml>
order allow,deny
allow from all
deny from XXXXXXXXX
RewriteCond %{HTTP_HOST} ^MYDOMAIN.info$ [OR]
RewriteCond %{HTTP_HOST} ^www.MYDOMAIN.info$
RewriteRule ^blog$ "http\:\/\/MYDOMAIN\.info\/category\/KEYWORD\-BLOG\/" [R=301,L]

Edited by andruha, 23 March 2010 - 11:26 PM.

#17 Randy


    Convert Me!

  • Moderator
  • 17,540 posts

Posted 24 March 2010 - 10:38 AM

Where is your default index page located? At mydomain.info/index.php, www.mydomain.info/index.php or at mydomain.info/blog/index.php or www.mydomain.info/blog/index.php?

And is it safe to assume the deny from lines are not triggering when Googlebot visits?

It's been awhile since I've tinkered with a WP .htaccess but I've always wondered why they use the %{REQUEST_FILENAME} !-f and !-d to suppress 404 errors instead of letting Apache's ErrorDocument simply serve a 404 Not Found. Here's is a commented version of those three lines explaining what they mean in something approximating English.

#If the request is not for a REAL file on the server
RewriteCond %{REQUEST_FILENAME} !-f
#and if the request is not for a REAL directory on the server
RewriteCond %{REQUEST_FILENAME} !-d
# Then serve the index file at the root level of the domain and stop processing rules
RewriteRule . /index.php

Make sense?

Now if there is no (www.)mydomain.info/index.php file because everything exists in the /blog/ subdirectory, the above is obviously going to send requests for non-existent files/directories to a page that the server in theory should throw a 404 error on. Which is okay, but can have some unintended effects if you're using other rewrites and the files being requested do not actually physically reside on the server.

Without digging into the nuts and bolts of it and conducting some tests of status codes for different addresses it's hard to say if this is having an unintended effect. But that said, if the root level of your site is actually in the /blog/ subdirectory you'd probably want the RewriteRule of this section to point to the /blog/ subdirectory (eg RewriteRule . /blog/index.php) instead of the root level location.

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!