Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!


Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 




From the folks who brought you High Rankings!

- - - - -

Potential 301 Problem - Duplicate Urls

  • Please log in to reply
3 replies to this topic

#1 Nozmo


    HR 2

  • Members
  • PipPip
  • 30 posts

Posted 06 May 2009 - 08:28 AM

To begin I am the site owner of <removed>. I am not a programer, but did all the content work on our site. Our updated site went live in Jan 2008. We have a method of converting dynamically generated URLs to what we call Friendly URLs. an example would be www.domain.com/Waterfront Real Estate. Prior to going live in 2008 our old "framed" site did very well in Google, MSN and Yahoo for all of our keywords. The new site has continued to do poorly in Google. We did encounter a problem in the second half of 2008 when we dropped for our primary keyword in Google, that we were being ranked for at position #3 to not in the top 1000. We discovered that our WMT showed all kinds of duplicate URLs - the dynamic ones and the friendly ones. We then created a robots.txt to prohibit crawling of the dynamic URLs. Overtime time the duplicates were removed from the WMT and we reappeared in our high position for the keyword <removed>. We made no improvement, however, in any other keywords in Google - but continued to do well in Y and M. Then last week we again disappeared from G, except for a miss-spelled version of our primary keyword.

This lead me to thinking that perhaps we did not do something correctly last year in creating the robots.tx. Maybe the redirects are not correct? This is what was done at that time:

301 redirect








301 redirect all requests for non €“www. URLs to the www version of the same URL

When 301 redirecting, always designate a URL as the location instead of doing this:

08/29/08 11:34:32 Browsing www.domain.com/info.php?info_id=119%20

Fetching www.domain.com/info.php?info_id=119 ...

GET /info.php?info_id=119 HTTP/1.1

Host: www.domain.com

Connection: close

User-Agent: Sam Spade 1.14

HTTP/1.1 301 Moved Permanently

Date: Fri, 29 Aug 2008 15:34:34 GMT

Server: Apache/2.0.52 (Red Hat)

X-Powered-By: PHP/4.4.9

location: MaineBeaches

Content-Length: 0

Connection: close

Content-Type: text/html

Ideally, all internal links to the home page should be coded to point to / or www.domain.com/

My question is - does this eliminate the possibility of duplicate URLs being seen by Google?Is there something else I should direct the programmers to look at? We also have a blog - www.domainNews.com that links to our primary website. I have been told that Google might be viewing this as a link farm and thus negatively influencing our site in some way. Others have told me that is not true. I also know our code is not pristine when view in a validator, but it is comparable to competitive sites in my area in terms of errors.

I am looking for a road map to correct any issues I might have, either technical or content, that is negatively affecting our results in the Google SERPS. I also ran an XENU report that I don't completely understand - especially the section on redirected URLs vs valid URLs. It is attached.

Thank you.

[Domain and keyword references removed per [url=http://www.highrankings.com/forum/index.php?act=boardrules]Forum Rules[/url]. Please read them.]

#2 Randy


    Convert Me!

  • Moderator
  • 17,540 posts

Posted 06 May 2009 - 09:40 AM

Frankly, you've got more issues that are going to be able to be dealt with in a forum post. Unless you already have a pretty good understanding of what needs to be done on the technical side of things you'd probably be better served by hiring someone with good knowledge on the technical side of SEO to conduct a site review and fix things. I have a feeling this is one of those cases where attempting to implement so-called Friendly URLs has gone badly because not all of the t's were crossed or i's dotted.

First things first. The redirect you show a trace on. It's really bad form to not have the full URL in the Location: field. I checked on your domain, and what you show in your trace is how it shows up today, with no url.

Second, though robots.txt can help to eliminate duplicate pages it does nothing to allow link popularity those old pages may have had to pass on to the new pages. So by simply excluding them but having no 301 redirect in the process you've effectively lowered your own link popularity.

So to fix things properly you not only need to fix the errors you had in your Xenu run, but you'll also need to go back in time and make sure calls to all of those old url addresses are being redirected (via 301 status) to the most appropriate new url. That's large job, and one that's not going to be solved for you from a forum post.

If you feel like taking it on yourself instead of hiring someone you first need to map out a plan, including all of the old urls and their new counterpart.

#3 Nozmo


    HR 2

  • Members
  • PipPip
  • 30 posts

Posted 06 May 2009 - 11:01 AM

Thanks for the comment. I am currently interviewing several technical people to help me with this problem - they will then work with the programmers to get the issues resolved. My question here was to ensure that I was on the right track with the URLs as being the primary problem. I wanted to generally understand that the code work that was done was not complete and that more work had to be done. You did not comment on the blog / website linking, but I don't really think this is my number 1 problem.

#4 Randy


    Convert Me!

  • Moderator
  • 17,540 posts

Posted 06 May 2009 - 12:09 PM

If the blog isn't a link farm then it's not a link farm. As long as it's a real resource wipe that one off of the potential problems.

On the rest, it's impossible to say if the url/redirection/duplicates is the root of the problem you've been seeing or not without knowing the whole history and doing a complete site review. It's a problem, but it's impossible to say if it's the root cause or a contributing factor without having all of the details.

That's why it would be smart to contract with someone who knows what to look for. They should be able to sort it out fairly expediently once they're armed with all of the data.

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

We are now a read-only forum.
No new posts or registrations allowed.