3 replies to this topic

[sergio]

Hello everyone,

recently I have received a link exchange request and as usual decided to check the page they were offering me. I was very surprised to see a man who was ready to let you have your link on his PR6 page When I typed in - domain.com/robots.txt - I saw a strange thing, I have never seen before. The line says: "(none) xx.xxx.x.xx 404". I put 'x' here in order not to unveil real IP - in origin there are real figures.

Does anyone know what this line means? How can we (SEOs) understand that from the "fair-play" point of view? And maybe some have other examples of strange robots.txt syntax?

Thank you all in advace

PS: I consider this page to be a fake and will not recommend to exchange links with ones of a kind. Normal pages should have normal things everywhere, even in robots.txt (especially, I would say)

[Randy]

I'm not sure I understand the question sergio. However if they're trying to control the server response per IP number in their robots.txt file, that's not something robots.txt is designed to do. It would require a very abnormal server set up for that to work as far as I know.

It almost sounds like they've confused robots.txt with .htaccess.

[sergio]

ye, Randy. It seems the same to me. Actually the question was - is it something tricky or normal? I think, there is sense to build some kind of a knowledge database about robots.txt tricks SEOs should be aware about. That would help to escape cheating by black SEOs, I think. In any case, thanks for your reply

[Jill]

It's neither tricky nor normal as it shouldn't have any effect on anything.