User-agent: * Crawl-Delay: 10 Allow: /requirements/fullcalendar Allow: /requirements/prettify Disallow: /admin/ Disallow: /tasks/ Disallow: /requirements/ Disallow: /config/