4 replies to this topic

[kslnor]

Here's hoping there are a few Wordpress pro's available...I use a plugin with my Wordpress blog that scans for problems as well as shows human visitors and crawlers.  I have noticed that it appears Googlebot crawls the same page multiple times in a day.  Said page indicated is the url followed by  ?pid=  then a string of numbers,  then a few words and symbols, then pg=25  The next item crawled is identical except the end where pg=26, the next one is pg=27, etc. 

 

Not sure what this means and why it looks like the same url is crawled multiple times. 

 

Does anyone have any clue what is going on ?

 

Thanks.

[Jill]

Are those additional URLs being indexed? I wouldn't worry so much about the crawling, but if they're being indexed, you could have a duplicate content problem happening. In which case you may need something like rel=canonical.

 

Also are those real URLs? The ones with the  pg=25, etc.? Do they exist on your site? If so, then you want them to be crawled (usually).

[chrishirst]

Does anyone have any clue what is going on ?

Google found a bunch of links that point to those URLs and decided to 'check them out'.

[qwerty]

If there are page numbers in the URLs, these are probably archives of posts for a particular category, tag, or time period. Most of the time, you'll want your content indexed as individual posts, each with their own URL, so it's usually wise to block indexation of some (and sometimes all) of your archive pages.

[kslnor]

Thanks for all help..

 

When I enter the URL in the Google search, the result is "Your search....did not match any documents". 

 

When I enter the URL in the address bar, I am taken to the post that has the URL without the ?pid= etc...

 

The ones with the pg=25...are not the real URL's and they don't exist on my site. 

 

Does this mean they are not indexed, therefore no worries?