# robots.txt for http://www.dictionaryproject.org User-agent: * Crawl-delay: 300 Disallow: /CMS/UI/DownloadPDF.aspx Disallow: /Donations/Backend/ Disallow: /Admin/ Disallow: /USA/ Disallow: /Assets/