Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!


Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 




From the folks who brought you High Rankings!


Disallow: /css/? Always Allowed Spiders Crawl Css

  • Please log in to reply
4 replies to this topic

#1 DJKay


    HR 5

  • Active Members
  • PipPipPipPipPip
  • 355 posts

Posted 24 August 2010 - 01:26 PM


I just had a developer ask me what benefit was there for letting a spider crawl CSS.

Quite honestly, the only things I have disallowed over the years was the scripting folder, subdirectories that contain duplicate content, duplicate landing pages used for various marketing promotions and other sub directories that have parts of the site that I do not want spidered.

I have always allowed them to crawl the CSS...did I make a mistake here....In regards to benefit, I always thought it had to do with the file names and the relationship to Htags, etc.

What is the answer? DJKay

#2 qwerty


    HR 10

  • Moderator
  • 8,695 posts
  • Location:Somerville, MA

Posted 24 August 2010 - 01:39 PM

I don't see much advantage in blocking access to the style sheet, except I suppose that keeping it from getting crawled might give one HTML page the opportunity to get crawled instead.

But I can definitely see a disadvantage to blocking it: doing so gives the search engine reason to question whether any of your formatting is serving the purpose of hiding content from users. Maybe that's just me being paranoid, but that's the first thing that I'd think of if I came across a blocked CSS.

#3 DJKay


    HR 5

  • Active Members
  • PipPipPipPipPip
  • 355 posts

Posted 24 August 2010 - 01:55 PM

Thanks, yes I know about the black hat issue..I also thought that because you can define the properties of the say..H1 and give greater clues to the engines in regards to semantically meaningful markup...don't know for sure...I don't code....

#4 Michael Martinez

Michael Martinez

    HR 10

  • Active Members
  • PipPipPipPipPipPipPipPipPipPip
  • 5,325 posts
  • Location:Georgia

Posted 24 August 2010 - 02:44 PM

The only legitimate reason to block access to CSS files is to manage server load, so far as I know. Most people don't need to worry about that.

Search engines say they would prefer to crawl the files, but there is no harm in blocking them.

If you're using CSS to do something sneaky and you block the search engine crawlers, you will still probably fail a manual review. Simply blocking crawlers from CSS directories should not result in a manual review but you have to remember that competitors might report a sneaky site.

That's about the worst that could happen, so far as I can see.

#5 Mooro


    HR 4

  • Active Members
  • PipPipPipPip
  • 157 posts
  • Location:Loughborough, Leicestershire

Posted 25 August 2010 - 03:25 AM

I want Google to see my style sheet, the names I give things relate to what they do, div_4 wrapping the top widgets isn't easy to grasp from a human point of view so I called it top_widgets further adding relevancy and making sure all elements of the site are singing the same tune.

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

We are now a read-only forum.
No new posts or registrations allowed.