# robots.txt for http://www.techtransfer.berkeley.edu/ # see http://www.robotstxt.org/ User-agent: * Disallow: /demo/ # password-protected Disallow: /coursematerials/ # password-protected Disallow: /freestuff/ # use /g3/ instead Disallow: /manual/ # nothing unique here Disallow: /media/ # use /videos/ instead Disallow: /multimedia/ # use /videos/ instead Disallow: /resources/ # use /videos/ instead Disallow: /resources/media/ # use /videos/ instead Disallow: /resources/media/multimedia/ # use /videos/ instead Disallow: /ped-safety/ # use /pedsafety/ instead Disallow: /ResourceGuide/ # use /pubs/ instead Disallow: /squirrelcart/ # honeypot Disallow: /train/ # use /training/ instead Disallow: /_vti_bin/ # honeypot Disallow: /weblogs/ # internal sysadmin use Disallow: /wwwlogs/ # internal sysadmin use