Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Google Can Index Inside Pages Which Are Not Pointing Link On Website?


  • Please log in to reply
18 replies to this topic

#1 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 04 October 2013 - 08:30 AM

We have online suppliers directory website which has approx. millions of pages & categories. On existing website all categories pages can be browsed through navigation & their category/sub-cate/sub-cate-1 etc.. But on redesign website we are not going to link each supplier page or category on the website through navigation, (we just display most popular one & some sidebar navigation) but all pages are live publicly on server or inside search. User can find those pages when hit any keyword in search.

 

My question is still will Google crawl & index each page on the website in-spite of each category page would not be linked back on the website?

 

Awaiting for your reply. 

 

 

 



#2 Alan Perkins

Alan Perkins

    Token male admin

  • Admin
  • 1,642 posts
  • Location:UK

Posted 04 October 2013 - 08:38 AM

 

My question is still will Google crawl & index each page on the website in-spite of each category page would not be linked back on the website?

 

No. You will need to do some work to ensure Google can crawl and index those category pages.

 

If the category URLs aren't changing then there's a chance Google will remember them and continue to index them for quite a while. But, over time, things will fall apart. You need to have links that Googlebot can see and follow if you want those category pages to be properly indexed.



#3 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 04 October 2013 - 08:50 AM

Thanks Alan for your quick reply. 

 

As we are redesigning so all page url will be redirected on new one. Its pretty difficult or structured all category on the website. So we main focus on online search that user search with keywords & find the page. We add all pages in xml sitemap. Search engine cant find from xml or crawl inside search?  



#4 torka

torka

    Vintage Babe

  • Moderator
  • 4,635 posts
  • Location:Triangle area, NC, USA, Earth (usually)

Posted 04 October 2013 - 09:09 AM

IMO, this is one of those situations where an XML sitemap might be useful. Google cannot fill out a search form to "find" pages. If you include the pages in your sitemap, Google should be able to "see" them. As Alan said, if the only way some pages can be accessed is via a search results page, Google will probably not index all of them... and perhaps eventually, all those "inacessible" pages could be dropped from the index. An XML sitemap should help prevent that from happening.

 

You might also want to consider adding a human-readable (HTML) sitemap to your site, if that's technically feasible. Not everyone likes to use search. Not everyone CAN use search. Some visitors may prefer to (or need to) browse. A sitemap will allow them to do this. It also means there will be links to all the pages which Google can follow to index them, even if something goes wrong with the XML sitemap.

 

--Torka :propeller:


Edited by torka, 04 October 2013 - 09:10 AM.

  • mansipadhya likes this

#5 Jill

Jill

    Recovering SEO

  • Admin
  • 32,982 posts

Posted 04 October 2013 - 09:30 AM

Site maps can only help so much. Because there will be relatively no internal link popularity to those pages, it's doubtful google will believe they're important. 



#6 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 04 October 2013 - 09:37 AM

Its convoluted to create human (HTML) sitemap for millions of category pages. Its a online directory of suppliers so our company is more emphasizing to make them searchable like kind of search engine. So what is the best option from an SEO perspective that we don't need to change the module & Google can crawl/index each page properly. I am not asking over n over  :think: but XML sitemap is the enough solution?



Site maps can only help so much. Because there will be relatively no internal link popularity to those pages, it's doubtful google will believe they're important. 

 

Yes this is also in our mind. On home page we have sidebar navigation with label of suppliers, manufacturing, wholesalers, some brand name etc also on company listing pages, we will point link to other supplement pages such as whitepapers, news, showroom etc. But not all category/sub cate page links as a internal links through navigation.  


Edited by mansipadhya, 04 October 2013 - 09:38 AM.


#7 Alan Perkins

Alan Perkins

    Token male admin

  • Admin
  • 1,642 posts
  • Location:UK

Posted 04 October 2013 - 09:54 AM

When I wrote "you will need to do some work", one of the "more work" options you have is to create and use an XML sitemap. Sitemaps are A solution, but ideally you can help this solution with some links on your website too.

 

E.g. if on your homepage you link to the most important 10 category pages, and on each of those 10 category pages you linked to 10 more, different sub-category pages, and so on then, before long, you would have linked to thousands and, ultimately, millions, of category pages. This, plus an XML sitemap, would be a good start ...


  • mansipadhya likes this

#8 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 04 October 2013 - 11:51 AM

 

When I wrote "you will need to do some work", one of the "more work" options you have is to create and use an XML sitemap. Sitemaps are A solution, but ideally you can help this solution with some links on your website too.

 

E.g. if on your homepage you link to the most important 10 category pages, and on each of those 10 category pages you linked to 10 more, different sub-category pages, and so on then, before long, you would have linked to thousands and, ultimately, millions, of category pages. This, plus an XML sitemap, would be a good start ...

 

 

Yes Alan we can link pages through top categories but there are so many so all are not sufficient under most 10 or 20. But if google can crawl anyway inside search pages (with rewrite url) so we don't need to point link all category pages on the website. Like on our existing website www.frasers.com, google indexed both page through navigation & inside search.

 

For example: if I browse "Acrylic Sheets" page will come /product-services/Acrylic+Sheets/p1/searchResults.jsf & if I hit the keyword by using search box, page will come productservices/frasers/all/Acrylic+Sheets/p1/searchResults.jsf

 (its duplicate content I know) but still google index both page. as we havn't listed inside search page in xml. so How google find these pages? we have also seen organic traffic through both pages. 



#9 Jill

Jill

    Recovering SEO

  • Admin
  • 32,982 posts

Posted 04 October 2013 - 12:10 PM

If you just want them indexed but don't care if they show up in searches, then your XML sitemap idea is fine. 



#10 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 04 October 2013 - 12:19 PM

Yes, but on our existing website all keyword search page has been indexed (google can't read inside search) & we haven't linked anywhere on the website so how google find them? 

 

Anybody has an idea about that? (this website is not in our host so we can't see)  



#11 bobmeetin

bobmeetin

    HR 6

  • Active Members
  • PipPipPipPipPipPip
  • 533 posts
  • Location:Colorado

Posted 06 October 2013 - 07:10 PM

If all your pages (findable only through your search utility) have been indexed on the old site, then there must be a function within the old CMS/eCommerce program that has created a site map or tree even if you don't know of it. Perhaps it's even a hidden facet of the search utility, don't know, just guessing.

 

Millions of pages is a lot for a sitemap, xml or html, to handle as a single page. If it were me, I'd create a PHP script to generate a category/subcategory/sub-sub/product tree that both human visitors and search engines could navigate as needed.



#12 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 06 October 2013 - 08:34 PM

So quick question; actually Google can index keyword search pages & search filters from any directory or eCommerce website?  Because I have seen couple of competitors website & they blocked their search it means might be their search url page is duplicate & that is why they blocked it. 



#13 Jill

Jill

    Recovering SEO

  • Admin
  • 32,982 posts

Posted 07 October 2013 - 07:27 AM

They can and do index pages from search. However it usually creates more problems than it helps, IMO. 


  • mansipadhya likes this

#14 mansipadhya

mansipadhya

    HR 3

  • Members
  • PipPipPip
  • 58 posts
  • Location:Toronto

Posted 07 October 2013 - 09:56 AM

Just want to understand, how Google index those pages from search? As we have those pages on the server under directory folder? that is why?



#15 Jill

Jill

    Recovering SEO

  • Admin
  • 32,982 posts

Posted 07 October 2013 - 12:25 PM

They literally put random words into your search box. It's kind of wild.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!