Best Answer qwerty , 01 May 2013 - 07:02 AM
Google will continue to request the URLs that 404, just to make sure they still 404, even if there aren't valuable links out there pointing to them. You would think that they would view this as a waste of crawling resources and would come up with a policy that, after a URL had responded with a 404 consistently for some period of time, they could stop requesting it unless they discovered a new link to that URL, but that doesn't appear to be the case. As far as I can tell, they don't stop checking.
That being said, they've been pretty clear in stating that 404s don't hurt you. You should make sure you don't link to them yourself, and that they don't appear on your XML sitemap, but apart from that, if you return a 404 response, your server is doing its job correctly. And the pages will drop out of the index. That is, they're not going to be returned for any queries -- even a query involving the site operator. So you're not likely to see requests for those pages from any user-agent other than a search engine spider.
So are you better off setting up 301 redirects? Maybe, but not necessarily. You've indicated that the pages you've deleted weren't valuable, apart from a few exceptions. The rest don't have authority or PR that could be funneled over to another page via a redirect or canonicalization. And since nobody but the bot is likely to request these pages, you're not improving the user experience by redirecting. So really, pretty much all you have to gain from redirecting is getting that list of 404 pages in Webmaster Tools to shrink.
If I'm wrong in assuming the pages have pretty much no value that could be passed on to another page, then maybe you have a little to gain from setting up redirects in this case. But if you do that, you want to avoid it getting out of hand. You don't want to end up with chains of redirects: you've deleted page A and redirected to B, and some time later you find that B has to go, and you redirect it to C, so now requests for A redirect to B and then redirect to C.Go to the full post