10 replies to this topic

[renken]

I'm wondering if a search engine can see the URL in an RSS feed. Or do search engines like RSS feeds because people are actually putting them into web sites and the search engines find them that way?

How exactly do search engines follow RSS feeds?

[Jill]

When someone posts a feed to their site, it creates links to the original content, which the search engines can follow in most cases.

[Scottie]

It's sort of a very structured sitemap to new content. It definitely is a nice thing to have. You can actually submit an RSS feed to Google Sitemaps if you want.

[randfish]

One of the great things about RSS feeds is their secondary ability to provide links. For example, if your feed gets re-published on many sites around the web, the links to the specific posts will be credited (in most cases) as deep links to your site's pages from the site reproducing your content - like article writing, this can be a great way to build link pop naturally. It makes sense for the SEs, too, as your feed wouldn't be copied all over the web unless those sites felt, editorially, that you were providing great, relevant content.

[renken]

I need to clarify. Is XML readable by a search engine spider?

If your XML file has:

<link>http://www.mylink.com</link>

Can the spider see that and follow it?

Or is the spider following the links that are generated by a parser? - Meaning there are actual links on a page that the parser creates out of the XML feed?

When you submit an XML feed to Google Site Maps - Google site maps generates the links in HTML format - so that the engine can then read actual html <a href...> links?

One more clarifying question.

Can a spider follow this link and find all the links within the feed?


<a title="RSS 2.0" href="http://www.e-healtha...topic=ALL"><img src="http://www.e-healthc...images/xml.gif" border=0 align="absmiddle"></a>

(if this were an actual live link on a web page)

[Jill]

They should be able to find and follow this one:

<a title="RSS 2.0" href="http://www.e-healtha...topic=ALL"><img src="http://www.e-healthc...images/xml.gif" border=0 align="absmiddle"></a>

But I don't believe they'd actually see the links within the resulting document as it shows up as just a text file.

I'm not up on this stuff enough, however, to be positive. I'm sure others here more familiar with XML will chime in!

[Scottie]

I remember Matt Cutts saying (possibly at SES last March) that Google can and did recognize any http:// domain.com text as a link, although they didn't guarantee to count it as a link.

To be sure, submit the RSS feed to Google Sitemaps which is designed to read XML content.

[renken]

Thanks, I added my URL into Google Site Maps.

According to the discussion that I had with our programmer, the search engines cannot index those links within XML because people can define XML in different ways. Therefore, a search engine would not identify <link>http://www.mylink.com</link> as an active link.

But my problem with that is why were people raving about XML feeds and how search engines pick them up? Were all these site webmasters that experienced with developing parsers at that time? Because what my programmer tells me is that the XML has to be parsed before the search engine can read it.

I was hoping that Michael Martinez would have a say about this - because I know he was one of the people talking about RSS feeds about a year or so ago.

[sweimh]

According to the discussion that I had with our programmer, the search engines cannot index those links within XML because people can define XML in different ways.  Therefore, a search engine would not identify <link>http://www.mylink.com</link> as an active link.

I don't think this is entirely true in terms of what you want to achieve here.

It's true that you can define XML in any ways you'd like, and SE probably won't recognize that. But if you are supplying your site links in RSS, your XML will be following a pre-defined format. So, you might want to have your programmer explore RSS instead of XML. (look for rss feeds in Yahoo News)

Also, Google Sitemap is in its own XML format, so it's not RSS. Google will recognize it, obviously, but other SE bots probably won't.

[Michael Martinez]

Google, Yahoo!, and MSN are all indexing and following XML/RSS content. They do it in different ways.

Google, for example, converts the content to link snippets and then includes them either in its news.google.com or its blogsearch.google.com service. I have not seen RSS feeds come up in anything other than those two specialized search tools' results. Other people may be able to make some suggestions.

Yahoo! will list the raw XML files in its search results, but naturally my browser won't display them when I click on the links. I think I've read that some browser add-ons will display these types of results.

MSN seems to be converting the XML data to link snippets but I could be mistaken on that account. I have not spent nearly as much time investigating MSN as I have the other two.

I don't know about Ask.

[renken]

Thanks - interesting stuff.

I'm more interested in getting the search engines to spider the actual articles than to display the XML/RSS feeds. So far, I don't see anything happening, though. The site that we have our articles on will not allow search engine spiders to find the actual pages where the articles reside (we are in the process of trying to fix that issue). However, I was hoping that the XML feeds would have led the spiders to the articles. But I guess that won't happen either.