# Robots.txt file for www.wadny.com # Robots that ignore this file will get their IP blocked # Good robots that behave are allowed in # Browse is blocked anyways because it can be highly dynamic and isn't useful for robots # If you run a valid, robots.txt-respecting robot and want to be allowed in, let me know # If I see something get robots.txt and then proceed to download stuff and it doesn't look like a user, it gets blocked User-agent: Googlebot User-agent: Slurp User-agent: Scooter User-agent: BlogPulse User-agent: Blogosphere User-agent: Wget User-agent: ia_archiver User-agent: fast Disallow: /browse.php Disallow: /browse/ # All other bots are blocked off. Screw them. User-agent: * Disallow: /