Ubuntuforums.org robots.txt:
Basically, robots that index ubuntuforums have access to anything that is availible to normal users.Code:User-agent: * Crawl-delay: 20
Ubuntuforums.org robots.txt:
Basically, robots that index ubuntuforums have access to anything that is availible to normal users.Code:User-agent: * Crawl-delay: 20
Bookmarks