Getting Pdfs Indexed When Registration Is Required Options

[ttw]

Hi Folks: I have a request from a client to put all their content-rich White Papers and Case Studies PDFs behind a registration form. The client's concern is that if someone comes to the website they must register - however, client wants the PDFs to get indexed.

What's the easiest way to achieve this?

[Randy]

What's the easiest way to achieve this?

You can't. Or I should say you can't without resorting to cloaking, which is something you really don't want to get involved in.

If you make the PDF's available for spidering, anybody who wants to read them can do so. Either via the cached version if you cloak 'em or by going directly to the pdf file.

About the best you can do is Excerpt part of the full document and make those available to any spider or use who hits the excerpt page(s), then require registration for the full version.

[MaKa]

Alternatively, you can have the registration form prominently and a small link (but visible) at the bottom saying, download PDF without registration.

+ No cloaking, fully indexed.
- People can find your PDF directly from the SEs without registration.

Could be interesting to do a test measuring which percentage of people actually register compared to the number of PDF downloads.

[ttw]

You can't. Or I should say you can't without resorting to cloaking, which is something you really don't want to get involved in.

If you make the PDF's available for spidering, anybody who wants to read them can do so. Either via the cached version if you cloak 'em or by going directly to the pdf file.

About the best you can do is Excerpt part of the full document and make those available to any spider or use who hits the excerpt page(s), then require registration for the full version.

Could I not build a separate page that contains all the links to the PDFs and then include this new page's link on the site map? I doubt most site visitors will see this page and will go through the registration process if they want to download one of the PDFs?

That chouldn't be considered cloaking.

[torka]

But if the PDF comes up in search (presumably the reason they want them indexed -- wanting them indexed but not wanting them to show up in search makes no sense), then anybody can read the PDF without registering by viewing Google's cache.

Allowing people to read the PDFs without registering defeats the purpose of the registration. Trust me, people like me who find the PDFs in the SERPs will read the interesting-sounding PDFs without registering. I don't care if they're formatted wierd or if they're missing pictures or what-not. I'm not registering if there's any way around it.

The ONLY way companies are gonna get any contact information out of me is if the PDF sounds irresistible and there's no other way to get at the content. Believe me, I will try any number of end runs to get around an apparent "required" registration. I get enough follow-up marketing as it is, and I don't care how many "we respect your privacy" reassurances anybody offers. I'm cynical and suspicious where that is concerned.

IMO, Randy's idea of an excerpt or teaser on the registration page is probably best from both a marketing and a search standpoint. Gives the SEs something to index without revealing the whole show. Gives the visitor a taste of what to expect and -- with any luck -- intrigues them enough to give up their contact info for registration. A win-win.

My (IMG:style_emoticons/default/penny.gif)

--Torka (IMG:style_emoticons/default/mf_prop.gif)

[ttw]

But if the PDF comes up in search (presumably the reason they want them indexed -- wanting them indexed but not wanting them to show up in search makes no sense), then anybody can read the PDF without registering by viewing Google's cache.

I don't care if people read the white papers because they found them in the search engines - to me that is all good. I want them indexed. The client simply wants to gather data if people first come to the site and then request a PDF - then they have to go through the annoying registration process.


So my question is, if I build a single page with all the PDFs and only link it in from the site map (where I doubt people will find it) I can't see how this can be cloaking.

Then, once people are already on the website, other than finding this site map link to the PDFs - they be required to register for the PDF links they find in the body content.

Would this work?

And thanks for your detailed response.

[MaKa]

Then, once people are already on the website, other than finding this site map link to the PDFs - they be required to register for the PDF links they find in the body content.

That should work and I do not think it will be seen as clocking. However, you will have only one link to each PDF. If you have a small link on the form you pleasantly surprise people as Torka and will probably end up with more links to each page linking to the PDFs. In addition you can measure how many people follow the do not register link with a simple onclick.

[Randy]

It would work and be safe as far as the engines are concerned Rosemary.

The problem being that if people were searching for the subject contained in the PDF file they're likely to get a link from the SERPs directly to the PDF file, unless you provide an abstract/excerpt elsewhere.

As a possible alternative, you can embed JavaScript into PDF files. So it's at least a possibility that you could embed a Username/Password query to the PDF files itself, each having a stock username/password in order to gain entry. I'm not sure if that would stop the bots from accessing them or not, but I'm guessing not since they're not JS enabled. It might be worth testing.

This post has been edited by Randy: Feb 13 2008, 07:01 AM

[torka]

The problem being that if people were searching for the subject contained in the PDF file they're likely to get a link from the SERPs directly to the PDF file, unless you provide an abstract/excerpt elsewhere.

Exactly my point. It's not going to be that "PDF map" page that comes up in the SERPs for the topics of the PDFs, it's probably going to be the PDFs themselves.

You'll get traffic directly to the PDFs, not to the rest of the site. Visitors will never see the registration form (or any of the other presumably valuable information on the site) in the first place.

If I can read what's in the PDF already, why would I bother going through the annoying registration process? Especially if I don't even know the annoying registration process exists in the first place...

--Torka (IMG:style_emoticons/default/mf_prop.gif)

[ttw]

If I can read what's in the PDF already, why would I bother going through the annoying registration process? Especially if I don't even know the annoying registration process exists in the first place...

--Torka (IMG:style_emoticons/default/mf_prop.gif)

Torka: It's really a tough issue about what's the best approach for our client in attracting the right type of prospects.

I believe there is more value in having all the PDFs indexed and available to searcher/prospects to locate in the SERPs. If the searcher/prospect finds the PDF in Google, reads the PDF and wants to know more - then they'll come to the site. Hopefully at that point they won't mind filling out a reg. form with real data if they want to download another PDF.

While I do suggest to our clients that they provide excerpts for each PDFs, a short excerpt doesn't come close to having the word power of a 20-page White Paper filled with all the technical jargon and phrases used in the industry.



Rosemary

[houndog]

I deal with this issue as well for a client - having valuable content in a PDF that requires registration. If you put all of the PDFs on one page, with the correct link that does not prompt login/registration, then I do have one more thought: would you organize them on pages in a way that puts related PDFs on one page, so that their copy and linked text makes that page stronger? Normal people won't be searching for exact phrases in a PDF, so might have a harder time finding these PDFs if they are on pages that aren't ranking well? Just a thought. Since you posted this in February, I wonder what your outcomes have been?

[ttw]

Hi Houndog:

I haven't seen a big change because the PDFs were already indexed before the registration requirement was put in place.

Rosemary