# www.robotstxt.org/ User-agent: BLEXBot Disallow: / # Allow crawling of all content User-agent: * Disallow: /not-found/ Disallow: /a/ Disallow: /r/ Disallow: /review/