User-agent: * Disallow: /admin/ # Don't crawl admin pages... Disallow: /member/ # ...or member pages (they're supposed to be private anyway) Disallow: /employer/ # ...or employer pages Disallow: /resume/ # ...or member resumes