# Allow all robots to spider everything by disallowing nothing User-agent: * Disallow: Crawl-Delay: 10 User-agent: * Disallow: /*source=