Robots.txt File Generator

Build standard Robots.txt instructions to manage crawl search bots. Direct spiders away from private folders.