# Expertsubmit - Strict Robots.txt User-agent: * Disallow: /app/ Disallow: /tmp/ Disallow: /admin/ Disallow: /*? Allow: /static/ Allow: /media/ Allow: /blog/ Allow: /case-studies/ Allow: /services/ Sitemap: https://expertsubmit.com/sitemap.xml # Block aggressive crawlers User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: /