My new client has literally hundreds and hundreds of inappropriate pages appearing in SERPs (uncovered using ScreamingFrog) such as:
- every blog post also appears with associated /feed/ file
- all their video lightbox files .js files appear
- there are separate URLs for every category in the blog
- Multiple css files are indexed
- js files that I can't even understand
- Many of the web pages have up to 5 URLs for each page which can be resolved with canonical tags.
- AND, every page also has an "https" version for no good reason.
The client uses Drupal as their CMS and I believe all these files are being generated by the CMS and that there must be an preferences or settings that can alter what files are generated. Is it possible with Drupal to set up a Robots.txt file to prevent these files from being generated?
Their blog categories are year/mo/date. I would like them to change these to a more search friendly structure. Is this possible in Drupal or is this going to create more work than the impact we might expect.
Any Drupal folks out there who could comment?
Edited by ttw, 31 May 2013 - 04:53 PM.