梦游时光互联客服中心 - 站长茶馆 - 求助：美国主机消耗CPU资源过多被封，请问怎么BAN掉多个蜘蛛啊，如百度&有道？

本帖主题: 求助：美国主机消耗CPU资源过多被封，请问怎么BAN掉多个蜘蛛啊，如百度&有道？

↓

SinaViVi

Friend

2009-8-17 09:39 AM No.1

发表人：An_Apple 主页个人Blog 社区短消息 Yahoo! TOP

An_Apple (Yeeher)
Discuz! user
Rank: 6

总裁俱乐部首席执行官

UID 3294
精华 2
积分 4670
帖子 4139
体力 4059
威望 49
贡献 1
阅读权限 60
注册 2006-11-27
来自上海市徐汇区
状态离线

[广告]: 备案信息核验单 - 现行国内备案必看

QUOTE:

Listing the top 40 browsers by the number of requests for pages, sorted by the number of requests for pages.

#reqs #pages browser
96706 91684 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
22701 20772 Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )
18222 17506 Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )
16632 13783 Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)
14247 13567 Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)
12178 9212 Mozilla/4.0 (compatible; MSIE 6.0)
10497 8465 Mozilla/5.0 (compatible; DotBot/1.1; http://www.dotnetdotcom.org/, crawler@dotnetdotcom.org)
7923 7821 Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+)
17683 7198 Mozilla/3.0 (compatible; Indy Library)
10247 7022 Sosospider+(+http://help.soso.com/webspider.htm)
7727 6753 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; CollapsarWEB qihoobot@qihoo.net)
11280 6747 Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/)
8029 5991 Baiduspider+(+http://www.baidu.jp/spider/)
7549 5745 Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)
6408 5637 Baiduspider+(+http://www.baidu.com/search/spider.htm)
6895 5517 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
6654 4980 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1; CollapsarDSP; )
4994 4494 Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
5816 4357 Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11; CollapsarTEXT
3190 2750 Mediapartners-Google
73385 2737 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
1804 1498 Mozilla/5.0 (compatible; MJ12bot/v1.2.4; http://www.majestic12.co.uk/bot.php?+)
13717 1322 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
3517 1234 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727)
2325 1077 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; InfoPath.2)
621 621 hitcrawler_0.1 ringken@gmail.com
780 557 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1; CollapsarGSCP )
13599 542 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; GTB6)
691 357 msnbot/2.0b (+http://search.msn.com/msnbot.htm)
382 345 psbot/0.1 (+http://www.picsearch.com/bot.html)
16712 301 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)
301 300 Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+)
7126 299 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) )
6828 290 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QQDownload 1.7)
9817 263 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MAXTHON 2.0)
246 222 al_viewer larbin2.6.3@unspecified.mail
511 219 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://bsalsa.com) )
640 212 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
303 196 Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
7476 183 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; TencentTraveler 4.0)
466276 15523 [not listed: 7,781 browsers]

[quote]

Listing the top 20 browsers by the number of requests for pages, sorted by the number of requests for pages.

# #reqs #pages browser
1 206526 175122 Netscape (compatible)
2 586409 47422 MSIE
  377429 40304 MSIE/6
  164787 5553 MSIE/7
  43018 1493 MSIE/8
  1087 51 MSIE/5
  74 11 MSIE/4
  14 10 MSIE/2
3 14554 13591 Mozilla
  136 2 Mozilla/1
4 14441 11628 Baiduspider+(+http:
  14441 11628 Baiduspider+(+http://www
5 10247 7022 Sosospider+(+http:
  10247 7022 Sosospider+(+http://help
6 11280 6747 Yeti
  11280 6747 Yeti/1
7 48017 6032 Firefox
  6226 4390 Firefox/1
  39596 1571 Firefox/3
  2193 71 Firefox/2
8 4994 4494 Sogou web spider
  4994 4494 Sogou web spider/4
9 3190 2750 Mediapartners-Google
10 621 621 hitcrawler_0.1 ringken@gmail.com
11 1016 496 msnbot
  691 357 msnbot/2
  325 139 msnbot/1
12 382 345 psbot
  382 345 psbot/0
13 9343 260 Safari
  6751 181 Safari/530
  1022 42 Safari/525
  849 19 Safari/531
  235 9 Safari/532
  319 8 Safari/528
  123 1 Safari/523
14 246 222 al_viewer larbin2.6.3@unspecified.mail
15 4328 211 Opera
  4032 123 Opera/9
  94 57 Opera/8
  18 18 Opera/7
  9 9 Opera/6
  175 4 Opera/10
16 142 127 ichiro
  142 127 ichiro/3
17 122 119 Trend Micro WTP Add-On 1.2.1046
18 129 116 li_viewer larbin2.6.3@unspecified.mail
19 607 113 Netscape
  595 109 Netscape/4
  3 3 Netscape/5
  5 1 Netscape/0
20 1694 101 Feedfetcher-Google; (+http:
  1694 101 Feedfetcher-Google; (+http://www
  4347 760 [not listed: 144 browsers] [quote]

[ 本帖最后由 An_Apple 于 2009-8-17 09:44 AM 编辑 ]

Hello, World! I'm Back!
2019.9.1