5 replies to this topic

[wholland]

Hi,

I have submitted my site to Google news to be included in their crawl and got the following response:

We reviewed however your site at www.missionbeachmagazine.com but, after some investigation, we've found that our system cannot crawl your site because one or more of the URLs on main pages is non-permanent. This prevents us from crawling the site for new content, as we are unable to detect the most current URL to be crawled.

For example, if a URL changes from /novembernews.html to /decembernews.html, Google will continue to crawl the novembernews.html page, and thus not find any new content.

We strive to include as many news sources as possible in Google News and would be happy to work with you to include your site in the future if you are able to change the URL-naming scheme.

Thank you for your interest in Google News.


I assume they are reffering to my "latest article" box on the right. Dose this mean if I want google news to crawl my site that I will have to remove this? Is there some thing I can do to change it so that it wil work for googles crawler?

Thanks,
Will

[RyanBlank]

i'm not very familiar with google news.. but couldn't you just specify an archive page or something similar that remains static as the page to crawl? keeping sure to update the page each time a new article is added.

[Jill]

Dose this mean if I want google news to crawl my site that I will have to remove this? Is there some thing I can do to change it so that it wil work for googles crawler?

That's exactly what it sounds like they were telling you.

It also sounds like all you need to do is create a permanent link for the articles and not change them. Seems like that would be easy enough, no?

[ewc21]

I find this article on the conditions of one news page to get --URL::65476c9030615d1f0553b174bced7a0f-- indexed by Yahoo! and Google.

I am skeptical about the Google explanation.

Any comments on this?

[wholland]

so it turns out my links are ok. I explained to google although my links may not be permanetly displayed on a certain page, that the pages being linked to by the links are permanent.

Google though that since a link was named for example November.php, that in december it would be removed and replaced with December.php. Which was not the case for my site, the November.php link would remain and a December.php would be added.

After sending an email explaining that, I got another email from google saying that was OK. But, I need to have a 3 digit number in the link. I already had the year in each link, that I included exactly for that reason. But google said the 3 digit number can not be the year. I renamed a folder in the path to a number and resubmitted.

Google news was happy with that and now crawls my site. I have searched google news and they have at least one of my articles crawled.

[Jill]

Good news, Will! Thanks for the update.