xiaop
V2EX  ›  站长

Sogou的蜘蛛怎么这么疯狂?堪比DDoS了

  •  
  •   xiaop · Dec 24, 2013 · 6123 views
    This topic created in 4556 days ago, the information mentioned may be changed or developed.
    66.249.64.1 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /redirect.php?goto=findpost&pid=567887 HTTP/1.0" 302 20 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.97 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/144666.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    220.181.89.164 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/6636/last/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.87 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /member/154806.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    61.135.189.124 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/167728.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.157 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/140326/p2.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.148 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/29477.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    66.249.64.2 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/178887.html HTTP/1.0" 200 7465 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.132 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/22570.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    66.249.64.4 - - [24/Dec/2013:23:10:57 +0800] m.quxizang.net "GET /bbs/t7730.html HTTP/1.0" 200 2151 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.154 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/119864.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    61.135.189.70 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/112342.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.152 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/29939/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    7 replies    1970-01-01 08:00:00 +08:00
    xiaop
        1
    xiaop  
    OP
       Dec 24, 2013
    其中一台小服务器已经挂掉,赶紧屏蔽了搜狗蜘蛛,日志是屏蔽后的。
    zhouzb889
        2
    zhouzb889  
       Dec 24, 2013
    曾经被这鬼东西消耗完流量,果断拒绝。
    它的Agent包括:Sogou web spider,Sogou News Spider,New-Sogou-Spider

    在DNS设置中解析到一个不存在的IP貌似也是可行的。

    另外,Sosospider也很疯狂!
    binux
        3
    binux  
       Dec 24, 2013
    又到年底覆盖率评估的时候了
    juicy
        4
    juicy  
       Dec 24, 2013
    。。。。我最近还刻意去添加了搜狗的SEO。。。。。悲剧。。。。
    katekiss
        5
    katekiss  
       Dec 24, 2013
    哎,不行就是不行
    solo7net
        6
    solo7net  
       Dec 25, 2013
    我的带宽被百度跑满过
    jjplay
        7
    jjplay  
       Dec 25, 2013
    藏ICP备很少见的啊,LZ西藏汉子啊
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   5321 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 52ms · UTC 08:23 · PVG 16:23 · LAX 01:23 · JFK 04:23
    ♥ Do have faith in what you're doing.