Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(404)

Issue 14761043: Sort urls, disallow crawling the /search interface (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
11 years, 11 months ago by M-A
Modified:
11 years, 11 months ago
Reviewers:
Andi
CC:
codereview-discuss_chromium.org
Visibility:
Public.

Description

Sort urls, disallow crawling the /search interface The fact the new /restricted urls in 789205957e8442952d5fb03feaf14512492e7ad7 wasn't in under the restricted section was an error. Google bot is on a spree crawling all random searches, which causes a fair amount of NeedIndexError errors, but Google bot doesn't care and continues. These pages do not need to be harvested anyhow to disable the whole /search section. TBR=albrecht.andi@gmail.com

Patch Set 1 #

Patch Set 2 : . #

Unified diffs Side-by-side diffs Delta from patch set Stats (+8 lines, -9 lines) Patch
M codereview/urls.py View 2 chunks +6 lines, -8 lines 0 comments Download
M static/robots.txt View 1 chunk +2 lines, -1 line 0 comments Download

Messages

Total messages: 2
M-A
11 years, 11 months ago (2013-10-16 20:47:59 UTC) #1
M-A
11 years, 11 months ago (2013-10-16 20:48:52 UTC) #2
Committed as 5f692edb9969.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f62528b