12 replies to this topic

[synergytheory]

I guess the title is self explanitory. However, if I use includes () with PHP [or SSI etc] to include a html file as the header [navi, header etc] do the sipders index the included file seperately from the main html doc, or both together as one file?

So when the algo does its thing, will it process the included html file and main html file together...or will it ignore the PHP script all together, or will it process the PHP and then the two html files seperately?

That sounds confusing to me and I wrote it But I am sure guys understand what I am saying.

Perhaps a better question would be, does includes() used for headers and footers hurt SEO, or improve it?

[Craig B]

if I use includes () with PHP [or SSI etc] to include a html file as the header [navi, header etc] do the sipders index the included file seperately from the main html doc, or both together as one file?

The spider indexes what is sees in the source, namely, the HTML. A spider (or anyone for that matter) cannot tell how a page is generated by just looking at the source file. We have some clues by looking at the page type (.php, .shtml, .pl etc.) but even that can be hidden at the server level. So answering your question, a spider indexes the page as one page, not as many components.

Perhaps a better question would be, does includes() used for headers and footers hurt SEO, or improve it?

It does neither. The only thing I would look for is the ability to change elements on a page-by-page basis. For example, you would want to be able to customize each <title> tag and not have the same title on every page on your website.

-Craig

[SearchRank]

Ditto what Craig said and I'll add that search engine spiders, crawlers, whatever you want to call them are like old browsers. Therefore when they index a page, they see what you and I see in our browser. They do not see the source code before a page is generated or assembled in the browser. So whatever code is in the include or its functions is assembled into a page once it is viewed in a browser. This is what the search engines see and index.

Having headers and footers as includes are great for ease of making changes to things - just never include title tags and meta description tags into those elements, otherwise you would have the same titles and meta description tags repeated page after page after page.

If your title and meta description tags are in include files, then a work around is to have a database or content management system that will allow you to create a unique title and meta description tag related to the specific content page of a site. We currently have a client who is trying to program something exactly like this because they have the opening and closing head tags and everything in between in their header include file.

[synergytheory]

I kinda thought that would be the case. But I wasn't sure if the spiders viewed includes() like a browser does i.e like one continuous html file [that's if you're including a html file]. I guess they really have no need to process any backend.

So one advantage may be, if you have a lot of javascript code in your header, and use includes () it would GREATLY cut down the amount of code the spiders have to search through before it gets to your copy?
Which brings up another questions do spiders have a time limit on how long they search a site before they move on or index it?

[Randy]

The time required varies a good bit from one search engine to another, and even within the same search engine depending upon a lot of factors out of your control. It's always a two step process though -- First the site gets spidered, then it gets added to the index. Two totally different things.

Generally speaking, what I normally see with Google on a new site is that Googlebot will come by to grab the index page and maybe a few others while it's there. Then anywhere from 1-3 weeks later it'll come back and deep crawl the site. Depending upon how many total pages are in the site it might get all of them on the first return visit, or it may have to come back several times.

When those pages start actually appearing in the SERPs can vary anywhere from a few days to two months. And there's not a thing you can do to affect G's time frame.

Do make sure you have at least one incoming link from a PR4 or above page though. They'll let you slide for a little bit, but if you have now incoming links at all, eventually the site will get dropped from the SERPs.

On the Title and Meta Description issue... Yes it's a very important factor to take into account when using an include for your header. Strictly speaking, you don't even need a CMS or Database to cure that problem though, and can still put all of the normal pre-Body tag info in an include file.

On smaller sites ( <500 pages), especially those which aren't already database driven, I tend to take a shortcut approach. Basically what I do is just variablize the Meta Description, Keyword and Title fields in the include file and then hard code those items on each page before I make the header include call.

Sounds cludgy maybe, but it works for me and is a piece of cake to work with. Plus this approach doesn't cause a db hit on the server every time a page is loaded. And better yet, I can tell at a glance of the page file what I'm targeting on that particular page rather than having to cross reference the database.

If that's not clear and you want an example give a yell.

[Ron Carnell]

In my experience, it's more like a three-step process.

1. Get spidered
2. Inclusion in Index
3. Stabilization in Index

Between #2 and #3, sites can disappear and rankings change drastically, nearly on a daily basis. A lot of people get scared, not realizing this is perfectly normal behavior. I suspect the cause is Google's multiple databases, but whatever the cause, the site will eventually stabilize. After that, rankings change little if at all ... until the next big shakeup.

[SearchRank]

So one advantage may be, if you have a lot of javascript code in your header, and use includes () it would GREATLY cut down the amount of code the spiders have to search through before it gets to your copy?

Actually if you have alot of JavaScript and put it in an include file, it will still be displayed in the html because remember that the server assembles all the code in the include into the page being viewed.

To keep lengthly JavaScript or CSS from appearing in the html of a page, you need to place these in their own files and then call for them. In the page you see in the browser, it would look like this to a spider and humans:

<link rel="stylesheet" type="text/css" href="text.css">
<script src="scripts/rollover.js" type="text/javascript"></script>
<script src="scripts/results_popup.js" type="text/javascript"></script>
<script src="scripts/certify.js" type="text/javascript"></script>

This example is taken from our site. The page calls for these things and then it all works in a browser but there is not all this lengthy code added to the html. An include is just that - a separate file that "Includes" its contents onto the page that calls for it.

[synergytheory]

Actually if you have alot of JavaScript and put it in an include file, it will still be displayed in the html because remember that the server assembles all the code in the include into the page being viewed.

To keep lengthly JavaScript or CSS from appearing in the html of a page, you need to place these in their own files and then call for them. In the page you see in the browser, it would look like this to a spider and humans:

<link rel="stylesheet" type="text/css" href="text.css">
<script src="scripts/rollover.js" type="text/javascript"></script>
<script src="scripts/results_popup.js" type="text/javascript"></script>
<script src="scripts/certify.js" type="text/javascript"></script>

This example is taken from our site. The page calls for these things and then it all works in a browser but there is not all this lengthy code added to the html. An include is just that - a separate file that "Includes" its contents onto the page that calls for it.

I do that with CSS but I never knew you could do that with javascript! Thanks for pointing that out.

But, I thought the crawlers and search engines would just see the script for includes as text, and not process it?
So it would just see:

<?
include('yourheader.html');
?>

and not the entire html header as an included file, as a browser does.

[Craig B]

This is HMTL...

<link rel="stylesheet" type="text/css" href="text.css">
<script src="scripts/rollover.js" type="text/javascript"></script>
<script src="scripts/results_popup.js" type="text/javascript"></script>
<script src="scripts/certify.js" type="text/javascript"></script>

This is not...

<?
include('yourheader.html');
?>

Spiders and browsers only read html. Although you code your page with PHP, it doesn't mean that we see it that way. When the server sees a page with the php extension, it parses and passes the php code to an application server (usually a apache module) and then the application server does stuff and passes back HTML to the user.

Does that make sense?

-Craig

[synergytheory]

Okay so the crawlers will read the included file EXACTLY like a browser reads it.

I think I am with it now thank you.

For some reason I kept thinking the spiders would read the html files as they sit in my server un-executed.

Edited by synergytheory, 19 December 2003 - 07:43 PM.

[SearchRank]

For some reason I kept thinking the spiders would read the html files as they sit in my server un-executed.

In order for them to do that, they would have to have FTP access to your server which they don't and then be able to grab the files off of there which they can't.

You are correct that spiders see what a browser sees. Select "view source" from your browser and look at the html code and that is exactly the same thing a browser sees.

Once a page loads in a browser, you no longer see the

<?
include('yourheader.html');
?>

but rather the html that is contained in the yourheader.html file.

[synergytheory]

Thanks searchrank, I still have no idea why I was thinking that way.

I will definitely put the javascript in it's own page now, like you suggested.

GREAT! so I can still impliment the includes for easy updates, and use my javascript on a seperate page to cut down the scripting.

Now all I need is a SEO friendly iframe and flash...

[niceguyeddie]

You are correct that spiders see what a browser sees. Select "view source" from your browser and look at the html code and that is exactly the same thing a browser sees.

They see what a text browser sees, not exactly what Mozilla or IE would display. There is a subtle difference there. Here is a decent Lynx viewer to check your pages with:

http://www.delorie.c...b/lynxview.html