DescriptionSort urls, disallow crawling the /search interface
The fact the new /restricted urls in 789205957e8442952d5fb03feaf14512492e7ad7
wasn't in under the restricted section was an error.
Google bot is on a spree crawling all random searches, which causes a fair
amount of NeedIndexError errors, but Google bot doesn't care and continues.
These pages do not need to be harvested anyhow to disable the whole /search
section.
TBR=albrecht.andi@gmail.com
Patch Set 1 #Patch Set 2 : . #
MessagesTotal messages: 2
|