What the robots.txt file does

https://www.openstreetmap.org/user/Carnildo/diary/390115