I have written a script for my affiliates system, generally speaking I want to disallow the indexing of the cgi-bin, but allow indexing of one particular script, as it does a 301 reditrect to the correct page and so want link-juice to be passed on.
if I have in my robots.txt file
but peoples websites will have links that include a query string, does just allowing indexing / accesses to the script on it's own also allow the SE's to follow the link.
or am i missunderstanding the robots.txt file, is following links and evaluating link juice nothing to do with the robots.txt and is that purely indexing.
do I even need to allow the base perl script in the robots.txt for the link juice 301 redirect mechanism to work?
Advice understanding this is appreciated.