1000 FAQs, 500 tutorials and explanatory videos. Here, there are only solutions!
Robots.txt file created by default
This guide gives information about the robots.txt file created by default for sites where this file is absent.
If the robots.txt file is absent from a site, a robots.txt file is generated automatically with the following instructions:
These instructions indicate to the robots put 10 second intervals between their requests, thus avoiding needlessly overloading the servers.