14 replies to this topic

[perkle]

Hi All
I would like to completely delete multiple pages in my site from all search engines. My question is what is the best way to do this correctly.

[Jill]

Just delete them from your server and make sure they provide a 404 http header response and eventually they'll disappear from the engines.

I am not sure, but there may also be a way to do that in your webmaster tools account, but I might be wrong about that.

[Randy]

You're right Jill. You can utilize the URL Removal tools most of the engines make available these days to speed up the process. However in order for the request to be honored the page(s) in question must first deliver a 404 Not Found response.

[perkle]

Just delete them from your server and make sure they provide a 404 http header response and eventually they'll disappear from the engines.

I am not sure, but there may also be a way to do that in your webmaster tools account, but I might be wrong about that.

Thanks Jill and randy
Who is they, and how do I know if They have a 404 http header response

[Jill]

"They" is your pages, well the old dead, URLs actually.

Run the URLs through a server header checker and you'll be able to see what the header response is.

[perkle]

"They" is your pages, well the old dead, URLs actually.

Run the URLs through a server header checker and you'll be able to see what the header response is.

OK bear with Jill, as I am dumber than a red brick.
I figured out what a checker was and it told me the following. Now what do I need to do, contact my website hosting company and have them add something.

HTTP/1.1 404 Not Found Date: Sun, 15 Jun 2008 20:19:13 GMT Server: Apache/2.0.52 (Red Hat) Content-Length: 321 Connection: close Content-Type: text/html; charset=iso-8859-1

[Randy]

Nope, you don't need to contact your host. The page is delivering exactly the status code message you want it to.

Look at the very beginning where it says 404 Not Found. That's what you want to see.

With that there you can now use the URL Removal tools some of the engines make available to get those pages truly dropped from the index more quickly.

[perkle]

I do it this way then.

Just for knowledge what would happen if I pasted the a rewriterule in my htaccess file.

[Jill]

Just for knowledge what would happen if I pasted the a rewriterule in my htaccess file.

???

[perkle]

I was asking about using the htaccess file to delete the files with RewriteRule ^

[don h]

Why would you want to do that? Do you want to redirect the URLs to a new page?

[perkle]

No I want the best way to delete them. I'll do as Jill and Randy said. I just wanted to know really why someone would advise me to use the htaccess file to do this. The forum software won't let me past the code I was given to do that.

[torka]

There's no reason to use the .htaccess file for anything in this case, if everything is as you've told us.

The files are already gone. The URLs for those missing files are already returning a 404-File Not Found header.

With those two things in place (files gone, 404 header), the listings will eventually drop out of the search engines all on their own. You don't have to do anything. The search engines will recognize that those URLs are returning a header that indicates the underlying file no longer exists.

If you want the listings to drop out of the search engine listings faster, you can use the facilities most of the major search engines offer to notify them these files are gone. In Google, you can do this through Webmaster Tools, for instance.

--Torka

[Randy]

I was asking about using the htaccess file to delete the files with RewriteRule

Taking a wild stab here, but I'm guessing someone has suggested you utilize a 301 Moved Permanently Redirect --as opposed to a 404 Not Found-- so that any link juice the old pages have would be passed on to another page of your site instead of being lost.

If these were main pages of your site that attract a lot of traffic and possibly inbound links from other sites, it might be a better strategy. However this isn't the question you asked. Those niggling little details are important.

The 404 will get them removed from the search engines. If new pages have basically taken the place of the old pages, a 301 might be more proper. Only you can answer this questioni, depending upon how important those old pages were to you and how well your new pages speak to those who might be looking for the content of each of your old pages.

[perkle]

Thanks Randy
No they are gone as you guys suggested. Your software here has some problem with me pasting the rewriterule with the Big G in brackets at the end indicating the file is gone and no forwarding address listed.

Thanks Torka
I have used the webmaster tools already and will try to find out how to do the same in Yahoo.

My REASON for this madness: The pages were part of a very nice directory for home builders which I ranked in the top 10 as a rule in Yahoo and others even better. But Big G in my webmaster tools account tells me my site is. ALLOWED Detected as a directory; specific files may have different restrictions

I have about 40 other pages not in the directory part that have been indexed but now only a few show the little green bar at all.

I just figured I'ed trash a great deal of work and traffic to my site and start over to see see if I can get any of my important pages to show in the SERP'S. Only my home page shows and its only forty first.

I have ask for two reconsideration request and no go. All of the directory pages are indexed but will not show up in the SERP's until about 500 or so.

I have worked hard on getting links to try to power up the directory and no go there either. Maybe there is a new rule out there (hidden to me) that tells others to stay away.