Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Advanced Crawler Optimization Methods


  • Please log in to reply
4 replies to this topic

#1 rsq1725

rsq1725

    HR 1

  • Members
  • Pip
  • 1 posts

Posted 24 January 2012 - 01:48 PM

I understand the basic tagging and on page SEO keyword related material.

I am looking for something more in the way the code is constructed or possible commands that can be given from the server or from more advanced programming so that pages can be crawled the best.


Any links, references, guidelines, articles, and subjects on what I am looking for would be very helpful and very much appreciated.


#2 Webnauts

Webnauts

    HR 3

  • Active Members
  • PipPipPip
  • 95 posts
  • Location:Germany

Posted 24 January 2012 - 06:36 PM

QUOTE(rsq1725 @ Jan 24 2012, 02:48 PM) View Post
I understand the basic tagging and on page SEO keyword related material.

I am looking for something more in the way the code is constructed or possible commands that can be given from the server or from more advanced programming so that pages can be crawled the best.
Any links, references, guidelines, articles, and subjects on what I am looking for would be very helpful and very much appreciated.


Implement "304 If_Modified-Since" HTTP headers.

#3 Michael Martinez

Michael Martinez

    HR 10

  • Active Members
  • PipPipPipPipPipPipPipPipPipPip
  • 5,082 posts
  • Location:Georgia

Posted 25 January 2012 - 05:47 AM

Put as few links on each page as possible. Many people screw up their crawl by creating site navigation that includes hundreds of links per page. It's better to leverage an HTML sitemap for crawling and let important pages feature/promote/highlight/draw attention to the latest or more important leaf nodes.

#4 Jill

Jill

    Recovering SEO

  • Admin
  • 32,926 posts

Posted 25 January 2012 - 09:06 AM

Make sure that any piece of content can only be accessed by one URL.

#5 SEOPeter

SEOPeter

    HR 1

  • Members
  • Pip
  • 2 posts
  • Location:Czech Republic, European Union

Posted 28 April 2012 - 08:05 AM

Use some cache plugin. It implements usually "304 Page Not Modified" HTTP status and takes care about "If-Modified-Since" HTTP headers. Your page load speed will fall. HTML code (without images) must be loaded under 500 ms.

Change your design to reduce size of images, scripts and css. Your whole page should be loaded under 2 seconds. Use Ctrl+F5 (Force reload) to test it.

Use Google Webmaster Tools and search for any 404 Page not found HTTP status.

Install Google "Page Speed" and achieve at least 95 / 100 points. There is great help in Google Page Speed.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!