Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Does Google Crawl Non-linked Urls?


  • Please log in to reply
3 replies to this topic

#1 UweKeim

UweKeim

    HR 1

  • Members
  • Pip
  • 5 posts
  • Location:Göppingen, Germany

Posted 07 July 2011 - 04:19 AM

Recently I discovered that pages of a competitor of us are in the Google index, although the URLs of the pages are not having any links pointing to them.

(I downloaded the complete site with WinHTTrack and really found no internal links)

The URLs that Google found and indexed are e.g.:

http://www.my-cool-website.com/somefolder/

On such URLs, no internal and no extern links are pointing to.

Instead, I found links to documents like:

http://www.my-cool-website.com/somefolder/some-document.pdf

I do not understand why non-refered documents are still being crawled by Google.

So my question is: Could anyone kindly explain me why non-refered URLs are found by Google?

What I do assume is that for each found URL, Google also moves up the folder hierarchy and crawls each found document, even if it is not refered directly.

Thanks
Uwe

#2 Jill

Jill

    Recovering SEO

  • Admin
  • 33,004 posts

Posted 07 July 2011 - 08:24 AM

Google finds urls in all kinds of crazy ways these days even if they're not linked to.

#3 UweKeim

UweKeim

    HR 1

  • Members
  • Pip
  • 5 posts
  • Location:Göppingen, Germany

Posted 07 July 2011 - 08:36 AM

Thanks a lot, Jill

#4 josephfranklynmcelroy

josephfranklynmcelroy

    HR 1

  • Members
  • Pip
  • 1 posts

Posted 08 July 2011 - 08:06 AM

Does the site have an xml sitemap? Pages to be indexed can be placed there... look for sitemap: tag in robots.txt or see if www.domain.com/sitemap.xml (or urllist.txt) exists

QUOTE(UweKeim @ Jul 7 2011, 04:19 AM) View Post
Recently I discovered that pages of a competitor of us are in the Google index, although the URLs of the pages are not having any links pointing to them.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!