# Google ad pages robots are special User-agent: Mediapartners-Google* Disallow: /cgi-bin/ Disallow: /discus/ Disallow: /email-addresses/ Disallow: /cartoons/read.cgi # Google robots get in free User-agent: Googlebot Disallow: /cgi-bin/ Disallow: /discus/ Disallow: /email-addresses/ Disallow: /cartoons/read.cgi # Ask.com robots get in free User-agent: Teoma* Disallow: /cgi-bin/ Disallow: /discus/ Disallow: /email-addresses/ # exclude Wayback User-agent: ia_archiver Disallow: / # exclude Entire system from robots User-agent: * Disallow: /