1 reply to this topic

[torka]

Normally I could probably figure this out, but its been a long week and I've been required to try to wrap my brain around so many different things I've run out of synapses to handle it all.

Let's say we're doing a direct mail campaign and we want custom landing pages to coordinate with each mailer. These landing pages are housed in subdirectories grouped under a main directory called mailers. I like this structure because we do a lot of direct mail marketing, and this keeps these landing pages all neatly segregated for the purposes of general site maintenance.

Let's say I have two subdirectories: mailer1 and mailer2, each of which contains a landing page for a different mailing piece. I've disallowed /mailers/ for all spiders in my robots.txt to prevent the subdirectories and their associated files from being indexed.

So far, so good.

However, on the mailer itself, we don't want to direct customers to www.oursite.com/mailers/mailer1 because that's too wordy and hard to type, so instead we use www.oursite.com/shortword and issue a 301 redirect via .htaccess to route visitors to the correct page.

So here's the question: Could the SEs index the page under the URL www.oursite.com/shortword because that URL isn't disallowed by robots.txt? Or would the fact the actual destination page of the 301 redirect (www.oursite.com/mailers/mailer1) is disallowed by robots.txt be enough to keep them out?

I'm uncharacteristically confused...

--Torka

[Alan Perkins]

So here's the question: Could the SEs index the page under the URL www.oursite.com/shortword because that URL isn't disallowed by robots.txt? Or would the fact the actual destination page of the 301 redirect (www.oursite.com/mailers/mailer1) is disallowed by robots.txt be enough to keep them out?

The URL www.oursite.com/shortword could be indexed (partially indexed page), but the content would not be indexed because that content doesn't sit at www.oursite.com/shortword but at www.oursite.com/mailers/mailer1, which would never be read. If the content cannot be read, it cannot be indexed.