28 replies to this topic

[dekyto]

dekyto are you sure yours isn't the dmoz title?

Jill,
Yes I am sure. The site isn't in ODP.

Do you have any idea why this would be coming up like this?
dektyo

[Michael Martinez]

In reading through this post I think that I am having a similar situation.
For years we have ranked at the top for 'Christmas Wreaths' the site is www.artificialchristmaswreaths.com

The title tag has been displayed as long as I can remember:
<title>BEAUTIFUL CHRISTMAS WREATHS by Artificial Christmas Wreaths.com</title>

Within the past week Google, Yahoo!, and Bing have stopped showing the title tag and are showing:
Google : Artificial Christmas Wreaths
Yahoo!: Artificial Wreaths
Bing: Artificial Wreaths

First of all, your robots.txt file is blocking all crawlers. Why is that?

However, it also appears that you have a canonical issue that may be related to the problem. Keep in mind that even though this might not have caused issues in the past, the search engines can and do change the way they do things all the time.

Your internal links for your root URL (home page) point to your HTTPS domain + "index.html", which is indexed in Google with the correct title. But when people search on your site, Google shows the root URL with a title (presumably) of its own design/extraction.

I don't see a listing for your site in Bing, except for the blog (which actually has the title "Artificial Christmas Wreaths" and displays properly) and a deep content page. UNLESS I search on the domain name. I think Bing has only partially indexed your site and they are probably using link anchor text from other sites to give your URL a title of "artificial wreaths".

Bing is pickier than Google and it's often more challenging to get Bing to index a site properly. You have to obtain links from sites that Bing indexes, and you appear to have done that, but for some reason Bing is not grabbing your content. Perhaps it's related to the canonical problem I mention above.

NOTE: I'm getting inconsistent hits on HTTPS vs HTTP for your site. I can definitely load the same content from both protocols.

[dekyto]

First of all, your robots.txt file is blocking all crawlers. Why is that?

However, it also appears that you have a canonical issue that may be related to the problem. Keep in mind that even though this might not have caused issues in the past, the search engines can and do change the way they do things all the time.

Your internal links for your root URL (home page) point to your HTTPS domain + "index.html", which is indexed in Google with the correct title. But when people search on your site, Google shows the root URL with a title (presumably) of its own design/extraction.

I don't see a listing for your site in Bing, except for the blog (which actually has the title "Artificial Christmas Wreaths" and displays properly) and a deep content page. UNLESS I search on the domain name. I think Bing has only partially indexed your site and they are probably using link anchor text from other sites to give your URL a title of "artificial wreaths".

Bing is pickier than Google and it's often more challenging to get Bing to index a site properly. You have to obtain links from sites that Bing indexes, and you appear to have done that, but for some reason Bing is not grabbing your content. Perhaps it's related to the canonical problem I mention above.

NOTE: I'm getting inconsistent hits on HTTPS vs HTTP for your site. I can definitely load the same content from both protocols.

Michael,
I don't know what is going on with the robots.txt file. I have never seen it before. I deleted it from the directory. It was dated 11/1/10. I have to check with my ISP and see what's going on with that.

I am sorry but I don't understand the root url information. I have links pointing to https AND http?

I am very concerned about this and will do whatever is required to correct it. Are the https and http link problems related to internal links that I have placed on the site or is a server configuration issue?

If you could clarify it for me so that I understand I would be extremely grateful.
Thanks.
dekyto

[dekyto]

First of all, your robots.txt file is blocking all crawlers. Why is that?

However, it also appears that you have a canonical issue that may be related to the problem. Keep in mind that even though this might not have caused issues in the past, the search engines can and do change the way they do things all the time.

Your internal links for your root URL (home page) point to your HTTPS domain + "index.html", which is indexed in Google with the correct title. But when people search on your site, Google shows the root URL with a title (presumably) of its own design/extraction.

I don't see a listing for your site in Bing, except for the blog (which actually has the title "Artificial Christmas Wreaths" and displays properly) and a deep content page. UNLESS I search on the domain name. I think Bing has only partially indexed your site and they are probably using link anchor text from other sites to give your URL a title of "artificial wreaths".

Bing is pickier than Google and it's often more challenging to get Bing to index a site properly. You have to obtain links from sites that Bing indexes, and you appear to have done that, but for some reason Bing is not grabbing your content. Perhaps it's related to the canonical problem I mention above.

NOTE: I'm getting inconsistent hits on HTTPS vs HTTP for your site. I can definitely load the same content from both protocols.

Michael,
I am a little puzzled by Bing. My index page shows up #1 for Christmas Wreaths, but without the title tag description.
I did a search for a specifically named wreath 'Christmas Magic Wreath' and it only gave the index page.
When doing a search for either term, Bing is automatically displaying Christmas Wreath images and 3 of the six that are displayed are our images.

Any help you can offer me regarding how to fix this would be greatly appreciated.
dekyto

[Michael Martinez]

I'm not trying to blow you off. It's just that I am very busy and I wasn't very satisfied with the feedback I was able to give you the other day. Also, you don't have enough posts here for a full Website review by [url=http://www.highrankings.com/forum/index.php?act=boardrules]Forum Rules[/url]. I may have unintentionally crossed the line with my reply to you.

I suggest you browse the forums and look for discussions that deal with ways of structuring Websites and canonicity. You need to become familiar with the terms we use and understand what they mean.

[Jill]

First of all, your robots.txt file is blocking all crawlers. Why is that?

That would explain the title then. Now that you've removed the robots.txt file things should come back correctly for you.

[dekyto]

Michael and Jill,
Thanks for everything! I have a lot to learn about this. You gave me a great starting point.
The main reason for my concern however is that this is a family business that has been steadily growing but something like what has happened could be really devastating. Especially at the beginning of the Christmas Wreath season.

A great point I would like to make is the robots.txt file. I NEVER CREATED IT. I would have never looked for it. I have contacted my ISP to see what happened. It was somehow created on November 1, 2010. I have had the site for almost 10 years without a robots.txt file. Without your help I would not have ever even considered such a thing. I truly appreciate your help.

I do have a lot to learn. But with regards to crossing the line with your reply, I appreciate it more than I can express.

I will research canonicity to more fully understand all of this.

Thank you for everything!
dekyto

[Jill]

A great point I would like to make is the robots.txt file. I NEVER CREATED IT.

Change your ftp passwords and stuff then, cuz someone is trying to sabotage you in the search engines. (That's my tinfoil hat explanation.)

Are you using wordpress or some other cms? Sometimes they have weird default settings that might have automatically added the robots.txt file.

[chrishirst]

More likely it's another case of a host being "helpful" by saving on data usage.

[dekyto]

Change your ftp passwords and stuff then, cuz someone is trying to sabotage you in the search engines. (That's my tinfoil hat explanation.)

Are you using wordpress or some other cms? Sometimes they have weird default settings that might have automatically added the robots.txt file.

Hi Jill,
I really appreciate the interest and follow ups!

I just found out what happened. One of the admins of my ISP put the robots.txt file into the directory on Nov. 1st. He thought that he had set it to ALLOW ALL not BLOCK ALL. Crazy!

My only hope now is that the search engines will re-index my pages quickly enough to salvage the tail end of the holiday season.

Thus far while my visitor counts have been down by slightly over 1/2, but our orders have only been down about 10-15%. We made a lot of improvements to both of our sites this year and one that is usually an under performer has been picking up the slack.

No matter what this is a terrible situation, but without yours (and Micheal's) help I WOULD HAVE NEVER LOOKED FOR A ROBOTS.TXT FILE! At least I am not banging my head against the wall trying to figure out what the problem is.

Thank you for everything!
dekyto

[Jill]

No matter what this is a terrible situation, but without yours (and Micheal's) help I WOULD HAVE NEVER LOOKED FOR A ROBOTS.TXT FILE! At least I am not banging my head against the wall trying to figure out what the problem is.

No problem! I'm just mad that Michael thought to check for it and I didn't.

Going to make this the forum thread of the week in the newsletter in case anyone else runs into similar problems they can be saved!

[Say Yebo]

I started making the robots.txt file and nofollow tag my first stop for troubleshooting indexin issues after I realized WP had some crazy default settings. It's saved me a lot of time ;-)

Now that you know what the problem was, theoretically it should be safe to remove the second instance of the keyword in the title, if you wanted to, and make use of another important word. But I see Google is giving the top organic positions to sites with double instances of the keyword in the title.

It never ceases to amaze me how they constantly tell us to do one thing and then reward the exact opposite behaviour. Also makes it harder to optimize the page successfully for more than one major word.

More and more I find myself having to choose between common sense and what actually works. It really shouldn't be this way should it?

[dekyto]

No problem! I'm just mad that Michael thought to check for it and I didn't.

Going to make this the forum thread of the week in the newsletter in case anyone else runs into similar problems they can be saved!

Jill,
It has only been 2 1/2 days (since removing the robots.txt file) and we already have our title tag displaying in the SERP's for the search of Christmas Wreaths.
Hopefully Bing/Yahoo will get a move on!

Thanks again so much to you and Michael!!
dekyto

[chrishirst]

More and more I find myself having to choose between common sense and what actually works. It really shouldn't be this way should it?

You have to do what works for YOUR pages.