SciPost Code Repository

Skip to content
Snippets Groups Projects
robots.txt 202 B
Newer Older
User-agent: AhrefsBot
Crawl-Delay: 10

Jean-Sébastien Caux's avatar
Jean-Sébastien Caux committed
User-agent: BLEXBot
Disallow: /
# Block Apple from training LLM models on our content
User-agent: Applebot-Extended 
Disallow: /

Jean-Sébastien Caux's avatar
Jean-Sébastien Caux committed
User-Agent: *
Disallow: /search