Are you a Google Analytics enthusiast?
More SEO Content
Number Of Pages In One Folder
Posted 04 December 2003 - 07:54 PM
I have heard that the max shouldn't be over 280, another suggested 5,000.
Is there anyone which has any experience with this?
Posted 04 December 2003 - 07:59 PM
Posted 04 December 2003 - 08:04 PM
Also, search engines have no problem indexing thousands of pages from the same active server page, and they don't even exist in a folder.
Posted 04 December 2003 - 09:34 PM
Most current operating systems, including Linux, store file details (name, size, pointer to disk block, etc) in a linked list. Every time a page is requested from that huge folder, the OS is reading, on average, N/2 entries in the list. I don't know where you heard the 280 number, but any modern computer could handle that with ease. The 5,000 number you heard is probably putting a strain on the system. And 31,000 entries in the same folder is really asking for trouble.
I would be concerned about performance in any situation, but I would be especially concerned with search engines. A spider can ask for pages much more quickly than your usual visitor, and each request is going to be putting an additional burden on the server. It probably won't really hurt any modern computer under any reasonable workload, but if you have ANY other weaknesses in the system a folder bursting at the seams just might put you over the edge.
The spider really won't care, or even know, that you have 31,000 files in one folder. It will care, however, if page requests start timing out or, heaven forbid, the computer should crash and burn. I think it's something at least worth watching ...
Posted 04 December 2003 - 09:36 PM
To date nothing has showed?
The site was already indexed pretty well for a year.
Posted 05 December 2003 - 05:47 AM
good points, I have a dedicated T1 the system is a raid5 scsi.
I know that when I open the folder in Dreamweaver it takes a while.
What's your suggestion <5000
Posted 05 December 2003 - 11:06 AM
Are the pages in that directory accessible via hyperlinks withing your indexed web site? You say they are dynamic. What does the URL to any of the given pages look like? Possibly you can provide an example without actually listing your URL?
The reason I'm raising this issue is that I added the directory in Mid-August. To date nothing has showed? The site was already indexed pretty well for a year.
Posted 06 December 2003 - 06:42 AM
You say they are dynamic
I think someone miss understood, they are static.
They are not individually linked from the root. The home page has a site map link, the site map links to a default page within that directory.
The 31,000 pages are static.
the url's are the product number---dirctory/7353663.htm
Posted 06 December 2003 - 07:34 AM
Does the default page within the directory link to any of the 31,000 pages in question? In other words, are there any links to these pages that a spider can follow, or do they only come up based on a search query or a form?
The home page has a site map link, the site map links to a default page within that directory.
Posted 06 December 2003 - 06:59 PM
I was asumming they would get spidered from the site map link to the default page within that big directory.
If these pages could get indexed we could really get some sales going.
Posted 07 December 2003 - 11:11 AM
Therefore if you 31,000 pages do not have links pointing to them in your web site, they will not get indexed. Period.
Posted 07 December 2003 - 12:38 PM
The only other thing I can think of would be to create a series of site maps, breaking everything down by category. But this would be a huge series, especially if you kept the number of links on each of those pages to a maximum of 100.
How do users access these pages now if there aren't any text links to them? Are they only available via a search?
Posted 07 December 2003 - 08:40 PM
Since those DB items can't be indexed I built each one a static page.
Posted 20 January 2004 - 06:26 AM
There *have* been instances where it appears that a search engine has found a page that has not been linked to, but the evidence is inconclusive -- that is, it's possible that someone somewhere linked to a particular page. Or that someone with the Google toolbar installed in his/her browser visited the page. Or ...?
I don't envy you having to implement so many links.
Posted 20 January 2004 - 04:28 PM
In the old DOS 6 days, (that covered up to Windows ME) once you went over 512 files in a folder no more could be added to that folder and you would start getting weird errors. We found this out by one user who installed way too many files in the root folder and the system would throw errors as it tried to write temp files to the root.
In NT4/2000 there is a limit of 65,536 files in a directory.
4,294,967,295 on NTFS5 (Wwindows 2000) max number of files on a system.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users