Zugriffe auf die robots.txt von http://www.christoph-bachner.net.

Aufzeichnung gestartet: 06.03.2004
Robots gesamt: 1296, Zugriffe: 53686

Robot Anzahl Prozent zuletzt
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) 25779 48.02% 12.11.2007 15:17:49
msnbot/1.0 (+http://search.msn.com/msnbot.htm) 7559 14.08% 12.11.2007 20:36:28
Googlebot/2.1 (+http://www.google.com/bot.html) 1190 2.22% 28.03.2006 11:54:31
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 1051 1.96% 12.11.2007 04:15:57
msnbot/0.3 (+http://search.msn.com/msnbot.htm) 926 1.72% 29.01.2005 22:08:05
JobSpider_BA/1.1 897 1.67% 13.01.2007 16:34:17
ia_archiver 826 1.54% 12.11.2007 21:07:03
msnbot-media/1.0 (+http://search.msn.com/msnbot.htm) 806 1.5% 09.11.2007 10:54:43
msnbot/0.11 (+http://search.msn.com/msnbot.htm) 665 1.24% 18.09.2004 01:38:40
Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html) 578 1.08% 12.11.2007 04:58:57
545 1.02% 05.11.2007 22:49:35
msnbot/0.9 (+http://search.msn.com/msnbot.htm) 522 0.97% 22.09.2006 01:30:23
TheSuBot/0.1 (www.thesubot.de) 350 0.65% 10.11.2006 00:59:42
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html) 304 0.57% 13.10.2007 02:01:32
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) 274 0.51% 12.11.2007 19:54:54
ia_archiver-web.archive.org 259 0.48% 08.11.2007 22:56:45
Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com) 256 0.48% 11.11.2007 11:59:36
Googlebot/2.1 (+http://www.googlebot.com/bot.html) 233 0.43% 04.08.2004 23:22:35
Gigabot/2.0 231 0.43% 23.03.2007 18:36:07
larbin_2.6.3 ([email protected]) 200 0.37% 11.11.2007 20:57:28
SurveyBot/2.3 (Whois Source) 182 0.34% 11.11.2007 23:31:13
MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+) 173 0.32% 26.10.2007 22:20:52
Mozilla/2.0 (compatible; Ask Jeeves/Teoma) 156 0.29% 10.11.2007 23:41:43
Gigabot/3.0 blogs.gigablast.com 152 0.28% 04.11.2005 05:14:46
Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.2 143 0.27% 08.11.2007 03:39:03
NutchCVS/0.05 (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected]) 138 0.26% 12.04.2006 21:16:35
psbot/0.1 (+http://www.picsearch.com/bot.html) 133 0.25% 10.10.2007 13:31:54
StackRambler/2.0 (MSIE incompatible) 128 0.24% 05.11.2007 20:31:31
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) 128 0.24% 12.11.2007 01:28:59
Mediapartners-Google/2.1 124 0.23% 13.02.2007 14:05:05
Gigabot/3.0 123 0.23% 15.09.2005 14:35:16
SBIder/0.8-dev (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html) 120 0.22% 28.09.2006 00:07:22
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) 101 0.19% 17.10.2007 00:53:38
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MSIECrawler) 100 0.19% 03.07.2007 18:59:54
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 98 0.18% 02.11.2006 12:47:19
Baiduspider+(+http://www.baidu.com/search/spider.htm) 95 0.18% 11.11.2007 12:16:43
Gigabot/2.0/gigablast.com/spider.html 94 0.18% 02.11.2006 19:58:39
Argus/1.1 (Nutch; http://www.simpy.com/bot.html; feedback at simpy dot com) 92 0.17% 04.03.2006 10:41:15
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (http://www.voila.com/) 92 0.17% 04.11.2007 03:11:40
voyager/1.0 91 0.17% 05.09.2007 02:02:02
Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/) 90 0.17% 01.09.2007 03:43:46
NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected]) 89 0.17% 12.07.2006 08:17:10
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MSIECrawler) 89 0.17% 01.07.2007 13:11:57
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; MSIECrawler) 82 0.15% 30.09.2007 21:20:32
appie 1.1 (www.walhello.com) 81 0.15% 28.09.2007 06:50:27
SiteSearch/2.0 (sitesearch.gigablast.com) 81 0.15% 24.05.2005 23:14:37
http://www.almaden.ibm.com/cs/crawler [fc2] 79 0.15% 16.05.2006 02:31:24
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 74 0.14% 24.10.2006 15:28:12
favorstarbot/1.0 (+http://favorstar.com/bot.html) 73 0.14% 09.11.2007 12:13:11
Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98) 69 0.13% 06.11.2007 15:42:12
Mozilla/4.7 [en] (X11; I; SunOS 5.8 sun4u) 69 0.13% 14.10.2006 14:03:36
sogou spider 69 0.13% 26.03.2007 23:31:39
envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html) 69 0.13% 11.11.2007 16:47:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) 66 0.12% 04.11.2007 07:21:59
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 66 0.12% 07.11.2006 20:54:39
Exabot/3.0 65 0.12% 15.01.2007 03:38:27
VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; [email protected]) 64 0.12% 31.10.2007 16:49:20
Mozilla/4.0 compatible FurlBot/Furl Search 2.0 (FurlBot; http://www.furl.net; [email protected]) 61 0.11% 24.09.2007 03:53:04
Mozilla/5.0 (compatible; Kyluka crawl; http://www.kyluka.com/crawl.html; [email protected]) 61 0.11% 06.04.2007 08:57:19
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 61 0.11% 05.08.2006 20:54:43
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Larbin/2.6.3 ([email protected]) 60 0.11% 21.03.2006 13:23:31
MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+) 60 0.11% 06.07.2006 06:47:29
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 59 0.11% 24.01.2007 10:56:47
NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 59 0.11% 18.06.2006 21:40:53
Mozilla/5.0 (compatible; Interseek/3.1) 57 0.11% 24.06.2007 23:38:55
Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at girafa dot com; http://www.girafa.com) 56 0.1% 06.11.2007 09:29:09
ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html) 56 0.1% 25.10.2007 07:14:14
SBIder/SBIder-0.8.2-dev (http://www.sitesell.com/sbider.html) 56 0.1% 09.11.2007 15:10:12
Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker ([email protected]; http://www.WISEnutbot.com) 54 0.1% 16.09.2006 11:21:06
Lsearch/sondeur 53 0.1% 29.10.2007 23:58:23
Jakarta Commons-HttpClient/3.0-rc4 52 0.1% 20.12.2005 04:50:46
Speedy Spider (Entireweb; Beta/1.0; http://www.entireweb.com/about/search_tech/speedyspider/) 52 0.1% 03.04.2007 09:27:11
Mozilla/5.0 (compatible; BecomeBot/3.0; +http://www.become.com/site_owners.html) 51 0.09% 08.11.2007 17:10:17
eagle 50 0.09% 03.06.2007 20:17:36
Snapbot/1.0 48 0.09% 11.04.2007 17:10:00
Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07) 48 0.09% 22.10.2007 09:04:21
Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/) 48 0.09% 12.11.2007 20:58:38
Gigabot/2.0; http://www.gigablast.com/spider.html 47 0.09% 07.10.2006 09:47:33
RedKernel WWW-Spider 2/0 (+http://www-spider.redkernel-softwares.com/) 46 0.09% 12.08.2006 14:31:13
Shim-Crawler(Mozilla-compatible; http://www.logos.ic.i.u-tokyo.ac.jp/crawler/; [email protected]) 44 0.08% 05.02.2007 16:39:08
Mozilla/5.0 (compatible; TridentSpider/3.1) 44 0.08% 12.06.2007 06:48:42
Spinne/2.0 Osttirol 43 0.08% 24.06.2004 21:19:16
NG/2.0 41 0.08% 11.10.2007 21:21:57
Mozilla/5.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml) 41 0.08% 09.11.2007 18:39:20
TurnitinBot/2.0 (http://www.turnitin.com/robot/crawlerinfo.html) 40 0.07% 10.07.2006 08:33:29
Googlebot-Image/1.0 40 0.07% 30.10.2007 03:20:32
Exabot/2.0 40 0.07% 20.08.2006 00:31:06
RedBot/redbot-1.0 (Rediff.com Crawler; redbot at rediff dot com) 40 0.07% 10.08.2007 11:50:53
IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler) 38 0.07% 23.07.2005 01:21:01
NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 38 0.07% 10.08.2006 13:33:56
Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/) 36 0.07% 28.03.2007 01:46:37
Gigabot/3.0 (http://www.gigablast.com/spider.html) 36 0.07% 05.11.2007 06:05:58
http://www.almaden.ibm.com/cs/crawler [bc3] 35 0.07% 29.12.2006 22:32:19
aipbot/1.0 (aipbot; http://www.aipbot.com; [email protected]) 35 0.07% 30.06.2006 19:32:51
Speedy Spider (Entireweb; Beta/1.1; http://www.entireweb.com/about/search_tech/speedyspider/) 35 0.07% 07.05.2007 20:10:47
Jetbot/1.0 33 0.06% 06.11.2004 18:48:28
Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.searchme.com/support/) 33 0.06% 16.10.2007 17:23:54
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; MSIECrawler) 31 0.06% 05.10.2007 17:41:40
ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl) 31 0.06% 09.11.2006 10:27:40
Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7.8) Gecko/20050512 Firefox/1.0.4 31 0.06% 08.11.2007 15:45:34
gamekitbot/1.0 (+http://www.uchoose.de/crawler/gamekitbot/) 30 0.06% 06.01.2007 12:22:53
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; MSIECrawler) 30 0.06% 22.12.2005 13:37:32
SeznamBot/1.0 (+http://fulltext.seznam.cz/) 30 0.06% 18.05.2006 00:49:52
iCCrawler (http://www.iccenter.net/bot.htm) 30 0.06% 12.07.2006 12:08:48
holmes/3.10.1 (OnetSzukaj/5.0; +http://szukaj.onet.pl) 30 0.06% 08.05.2007 07:22:27
Gigabot/1.0 29 0.05% 03.10.2007 18:37:21
gonzo1[P] mailto:[email protected] 29 0.05% 11.09.2006 12:20:21
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 29 0.05% 20.08.2006 14:56:57
updated/0.1beta (updated.com; http://www.updated.com; [email protected]) 28 0.05% 23.09.2005 05:45:21
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 28 0.05% 12.05.2006 13:54:37
bot/1.0 (bot; http://; [email protected]) 28 0.05% 01.09.2007 04:24:01
Pingdom GIGRIB (http://www.pingdom.com) 28 0.05% 12.11.2007 03:46:50
WIRE/0.11 (Linux; i686; Robot,Spider,Crawler,[email protected]) 27 0.05% 02.07.2006 02:17:52
Mozilla/4.0 26 0.05% 13.05.2007 08:45:55
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.10) Gecko/20050724 Firefox/1.0.6 25 0.05% 07.10.2007 03:52:14
IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler) 25 0.05% 29.07.2007 05:41:47
Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; ) 25 0.05% 31.10.2007 02:46:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.9 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; [email protected]) 25 0.05% 23.04.2007 20:37:23
Patwebbot (http://www.herz-power.de/technik.html) 24 0.04% 20.05.2006 01:38:21
Speedy Spider (Beta/1.0; [email protected]) 24 0.04% 17.04.2005 21:36:28
Sensis Web Crawler (search_commentsatsensisdotcomdotau) 24 0.04% 31.05.2007 08:41:22
Speedy Spider (Entireweb; Beta/1.3; http://www.entireweb.com/about/search_tech/speedyspider/) 24 0.04% 28.05.2007 09:30:51
Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html) 23 0.04% 13.04.2004 22:34:42
"Mozilla/5.0 (X11; U; Linux i686;en-US; rv:1.2.1) Gecko/20030225" 23 0.04% 23.02.2006 03:44:15
IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler) 23 0.04% 20.07.2006 06:49:27
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MSIECrawler) 23 0.04% 12.05.2007 00:23:34
sproose/1.0beta (sproose bot; http://www.sproose.com/bot.html; [email protected]) 23 0.04% 17.03.2007 05:21:23
CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; [email protected]) 23 0.04% 06.06.2007 04:26:20
MSRBOT (http://research.microsoft.com/research/sv/msrbot/ 23 0.04% 08.11.2007 22:12:31
NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 22 0.04% 04.06.2007 06:15:35
MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+) 22 0.04% 16.01.2007 18:09:18
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322) 22 0.04% 22.09.2006 14:26:05
Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html) 22 0.04% 07.11.2007 09:20:03
CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; [email protected]) 21 0.04% 09.11.2007 05:10:25
Vagabondo/3.0 (webagent at wise-guys dot nl) 20 0.04% 12.05.2006 14:06:16
disco/Nutch-1.0-dev (experimental crawler; www.discoveryengine.com; [email protected]) 20 0.04% 09.11.2007 01:35:06
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 19 0.04% 18.05.2004 20:06:04
Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html) 19 0.04% 09.02.2005 02:20:40
Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot .php) 19 0.04% 27.01.2006 05:19:10
Speedy Spider (Beta/1.0; www.entireweb.com) 19 0.04% 01.09.2006 23:12:08
Mozilla/4.0 (compatible;) 19 0.04% 26.01.2006 14:04:12
Gigabot/2.0 (http://www.gigablast.com/spider.html) 19 0.04% 13.10.2007 21:43:33
holmes/3.11 (http://morfeo.centrum.cz/bot) 19 0.04% 02.11.2007 23:04:14
NaverBot-1.0 (NHN Corp. / +82-2-3011-1954 / [email protected]) 18 0.03% 20.06.2005 04:09:20
telnet0.1 ([email protected]) 18 0.03% 18.05.2005 20:32:39
NaverBot-1.0 (NHN Corp. / +82-31-784-1989 / [email protected]) 18 0.03% 17.09.2006 14:59:21
del.icio.us-thumbnails/1.0 Mozilla/5.0 (compatible; Konqueror/3.4; FreeBSD) KHTML/3.4.2 (like Gecko) 18 0.03% 23.08.2007 06:26:16
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312462) 18 0.03% 21.01.2007 22:11:14
WebAlta Crawler/1.3.33 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) 18 0.03% 03.07.2007 06:36:08
PHPCrawl 18 0.03% 31.10.2007 05:44:14
larbin_2.6.3 ([email protected]) 17 0.03% 05.10.2005 22:53:39
Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 17 0.03% 14.06.2006 23:29:05
Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 17 0.03% 12.09.2006 11:03:44
visit_www.yourpoll.net ([email protected]) 17 0.03% 30.07.2007 15:29:31
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; MSIECrawler) 16 0.03% 01.08.2006 16:31:10
lmspider ([email protected]) 16 0.03% 24.07.2005 02:01:10
Mozilla/4.0 Accept: */* Accept-Language: en-us, en, de;q=0.50 16 0.03% 27.03.2005 04:34:38
cfetch/1.0 16 0.03% 21.11.2005 18:07:28
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8) Gecko/20051111 Firefox/1.5 16 0.03% 08.08.2006 23:13:11
MSRBOT 16 0.03% 06.04.2007 20:25:36
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7 16 0.03% 23.09.2007 10:45:04
sproose/0.1 (sproose bot; http://www.sproose.com/bot.html; [email protected]) 16 0.03% 22.09.2006 09:42:34
ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl(at)ml(dot)nict(dot)go(dot)jp) 16 0.03% 23.05.2007 01:57:54
Ocelli/1.3 (http://www.globalspec.com/Ocelli) 15 0.03% 12.08.2007 08:57:08
Python-urllib/1.15 15 0.03% 07.03.2007 07:13:02
FAST MetaWeb Crawler (helpdesk at fastsearch dot com) 15 0.03% 30.09.2007 08:17:05
OrangeSpider 15 0.03% 23.08.2007 00:54:30
ichiro/1.0 ([email protected]) 14 0.03% 27.09.2005 16:59:31
NetResearchServer/3.5(loopimprovements.com/robot.html) 14 0.03% 10.12.2005 09:25:42
Twiceler www.cuill.com/robots.html 14 0.03% 21.05.2006 04:31:16
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 14 0.03% 17.05.2006 23:43:23
FyberSpider 14 0.03% 30.12.2005 23:03:11
Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://corp.krugle.com/crawler/info.html; [email protected]) 14 0.03% 23.10.2007 20:33:11
MSRBOT (http://research.microsoft.com/research/sv/msrbot) 14 0.03% 26.05.2007 17:10:02
Novosoft Spider/Nutch-0.9 (http://www.novosoft.net) 14 0.03% 21.10.2007 12:20:23
Pompos/1.3 http://dir.com/pompos.html 13 0.02% 24.10.2005 17:17:41
Python-urllib/1.16 13 0.02% 25.07.2006 01:47:27
http://www.almaden.ibm.com/cs/crawler [c11] 13 0.02% 01.08.2005 11:43:48
snap.com beta crawler v0 13 0.02% 28.01.2006 18:59:34
Mozilla/5.0 (compatible; OnetSzukaj/5.0; +http://szukaj.onet.pl) 13 0.02% 31.08.2006 10:55:38
IconSurf/2.0 favicon monitor (see http://iconsurf.com/robot.html) 13 0.02% 16.10.2006 21:32:00
findlinks/1.1-a7 (+http://wortschatz.uni-leipzig.de/findlinks/) 13 0.02% 24.01.2006 12:28:17
findlinks/1.1-a8 (+http://wortschatz.uni-leipzig.de/findlinks/) 13 0.02% 27.01.2006 02:55:19
noxtrumbot/1.0 ([email protected]) 13 0.02% 27.01.2007 17:48:28
Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/rob ot.php) 13 0.02% 17.10.2007 05:33:38
BilgiBot/1.0(beta) (bilgi.com(beta); http://lucene.apache.org/nutch/bot.html; [email protected]) 13 0.02% 02.10.2006 17:46:12
MQBOT/Nutch-0.9-dev (MQBOT Crawler; http://falcon.cs.uiuc.edu; [email protected]) 13 0.02% 26.01.2007 13:17:04
http://www.almaden.ibm.com/cs/crawler [c01] 12 0.02% 07.09.2004 19:04:03
QuepasaCreep ( [email protected] ) 12 0.02% 15.06.2004 04:50:29
search.ch V1.4.2 ([email protected]; http://www.search.ch) 12 0.02% 15.06.2005 04:10:25
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) 12 0.02% 27.11.2005 17:49:59
Holmes/1.0 12 0.02% 28.03.2005 12:27:34
envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.php) 12 0.02% 30.09.2006 01:54:34
WebCrawler_1.1 ([email protected]) 12 0.02% 08.03.2007 14:05:25
LarbinWebCrawler ([email protected]) 12 0.02% 08.04.2007 01:51:01
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; MSIECrawler) 11 0.02% 26.12.2004 21:08:37
ScSpider/0.2 11 0.02% 13.09.2007 20:15:48
geniebot ([email protected]) 11 0.02% 24.11.2005 06:27:18
Mozilla/3.0 (Compatible;Viking/1.9) (Viking/1.9) (prefetch) 11 0.02% 01.11.2007 07:45:07
MJ12bot/v1.0.5 (http://majestic12.co.uk/bot.php?+) 11 0.02% 23.12.2005 22:29:01
IlTrovatore-Setaccio/1.2 (Italy search engine; http://www.iltrovatore.it/bot.html; [email protected]) 11 0.02% 25.11.2005 10:11:44
xirq/0.1-beta (xirq; http://www.xirq.com; [email protected]) 11 0.02% 16.12.2005 17:34:21
NetResearchServer/4.0(loopimprovements.com/robot.html) 11 0.02% 21.01.2007 10:28:37
schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; +http://www.schibstedsok.no/bot/) 11 0.02% 07.08.2007 12:29:14
stalker/0.1 (http://www.stalkpire.de/bot.aspx) 11 0.02% 27.04.2007 23:37:08
Nutch-test/Nutch-0.9-dev 11 0.02% 11.09.2006 07:17:38
Snapbot/1.0 (+http://www.snap.com) 11 0.02% 20.06.2007 12:43:03
Mozilla/5.0 (compatible; Synoobot/0.9; http://www.synoo.com/search/bot.html) 11 0.02% 22.05.2007 01:27:17
SeznamBot/1.0 (+http://fulltext2.seznam.cz/) 11 0.02% 01.09.2007 09:55:18
VadixBot 11 0.02% 04.11.2007 01:54:40
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Internet Explorer 6.0 powered by Celos Computer GmbH; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 10 0.02% 28.12.2004 14:27:47
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 10 0.02% 30.01.2005 22:11:55
Crawler ([email protected]) 10 0.02% 21.06.2005 01:44:49
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 10 0.02% 01.05.2005 09:25:05
Mozilla/4.0 (compatible; grub-client-2.6.0) 10 0.02% 01.07.2005 04:48:28
Minuteman/0.4 10 0.02% 03.10.2005 12:36:06
page_verifier (http://www.securecomputing.com/goto/pv) 10 0.02% 30.10.2007 02:32:35
larbin_2.6.3 ([email protected]) 10 0.02% 12.07.2007 10:18:42
Nutch Test/Something strange (Nutch Test; http://www.mistral.com) 10 0.02% 06.03.2007 14:55:47
Mozilla/5.0 (compatible; heritrix/1.10.2 +http://i.stanford.edu/) 10 0.02% 06.06.2007 14:09:18
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 9 0.02% 01.08.2004 14:35:39
LWP::Simple/5.76 9 0.02% 31.01.2005 19:11:25
Iltrovatore-Setaccio/1.2 (It-bot; http://www.iltrovatore.it/bot.html; [email protected]) 9 0.02% 12.01.2005 04:14:11
Vagabondo/2.0 MT 9 0.02% 21.05.2005 03:57:07
Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot .php) 9 0.02% 24.08.2005 16:30:22
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) 9 0.02% 25.06.2006 12:55:42
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Accept: */* Accept-Language: en-us, en, de;q=0.50 9 0.02% 26.04.2005 02:07:35
Accoona-AI-Agent/1.1.1 (crawler at accoona dot com) 9 0.02% 22.08.2006 00:24:20
findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/) 9 0.02% 19.12.2005 06:23:11
NutchCVS/0.8-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; [email protected]) 9 0.02% 13.07.2006 12:53:58
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050720 Fedora/1.0.6-1.1.fc3 Firefox/1.0.6 9 0.02% 22.04.2006 06:45:15
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 9 0.02% 16.07.2006 11:37:48
psycheclone 9 0.02% 11.07.2006 20:09:31
TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html) 9 0.02% 23.08.2007 08:17:34
Firefox ([email protected]) 9 0.02% 02.10.2006 09:28:05
MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; [email protected]) 9 0.02% 30.01.2007 15:50:25
onCHECK-Robot, www.onsearch.de 9 0.02% 11.08.2007 11:52:51
MJ12bot/v1.1.2 (http://majestic12.co.uk/bot.php?+) 9 0.02% 15.02.2007 09:01:44
CazoodleBot/0.1 (CazoodleBot Crawler; http://www.cazoodle.com; [email protected]) 9 0.02% 16.06.2007 19:43:23
MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; [email protected]) 9 0.02% 21.06.2007 20:04:44
htdig/3.1.5 ([email protected]) 9 0.02% 20.06.2007 05:44:25
CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; [email protected]) 9 0.02% 31.07.2007 00:45:22
GOFORITBOT ( http://www.goforit.com/about/ ) 8 0.01% 04.03.2007 02:37:44
NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 8 0.01% 12.04.2006 07:58:25
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 8 0.01% 20.03.2006 19:41:24
W3C-checklink/4.2.1 [4.21] libwww-perl/5.803 8 0.01% 16.10.2006 12:04:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Avant Browser; MSIECrawler) 8 0.01% 10.05.2006 07:21:44
NG-Search/0.90 (NG-SearchBot; http://www.ng-search.com; ) 8 0.01% 11.09.2006 22:05:48
asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com) 8 0.01% 12.08.2006 06:32:31
ilial/Nutch-0.9-dev 8 0.01% 21.10.2006 13:08:39
Nutch0.9/Nutch-0.9-dev 8 0.01% 22.01.2007 15:35:21
GiNDev/Nutch-0.9 (Germanistik im Netz; http://www.germanistik-im-netz.de/gin-www-search/gin-www-search-about. html; D dot Eichner at ub dot uni-frankfurt dot de) 8 0.01% 18.10.2007 15:36:27
Java/1.6.0_01-ea 8 0.01% 24.06.2007 11:06:52
Yeti/0.01 (nhn/1noon, [email protected], check robots.txt daily and follow it) 8 0.01% 06.11.2007 11:56:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) 7 0.01% 27.07.2006 16:11:18
Amfibibot/0.06 (Amfibi Robot; http://www.amfibi.com; [email protected]) 7 0.01% 23.04.2005 18:27:03
CydralSpider/1.9 (Cydral Web Image Search; http://www.cydral.com) 7 0.01% 02.07.2005 22:48:02
Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; [email protected]) 7 0.01% 19.11.2005 05:53:40
Combine/2.0 7 0.01% 24.06.2006 09:23:00
eagle Accept: */* Accept-Language: en-us, en, de;q=0.50 7 0.01% 07.08.2005 15:49:23
Miva ([email protected]) 7 0.01% 21.03.2006 01:17:33
BigCliqueBOT/1.03-dev (bigclicbot; http://www.bigclique.com; [email protected]) 7 0.01% 17.08.2005 14:55:17
Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/rob ot.php) 7 0.01% 30.11.2005 03:49:11
ichiro/2.0 ([email protected]) 7 0.01% 14.12.2005 13:19:47
Exabot-Images/1.0 7 0.01% 01.07.2006 01:18:02
Mozilla/5.0 (compatible; Sokitomi crawl; http://www.sokitomi.com/crawl.html; [email protected]) 7 0.01% 19.05.2006 08:49:01
bot/1.0 7 0.01% 15.05.2006 11:19:15
WIRE/0.11 (Linux; i686; Bot,Robot,Spider,Crawler,[email protected]) 7 0.01% 02.06.2006 18:15:04
Robo Crawler 6.4.5 ([email protected]) 7 0.01% 02.07.2006 23:10:24
SiteSuBot 7 0.01% 15.06.2006 17:07:18
Nutch0.8/Nutch-0.9-dev 7 0.01% 20.01.2007 17:44:09
ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot) 7 0.01% 15.04.2007 00:34:20
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; MSIECrawler) 7 0.01% 10.07.2007 13:58:05
www.fi crawler, contact [email protected] 7 0.01% 04.09.2007 21:24:13
ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; [email protected]) 7 0.01% 01.06.2007 20:45:26
great-plains-web-spider/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected]) 7 0.01% 24.09.2007 02:56:43
holmes/3.11 (OnetSzukaj/5.0; +http://szukaj.onet.pl) 7 0.01% 02.11.2007 22:55:37
Ocelli/1.4 (http://www.globalspec.com/Ocelli) 7 0.01% 06.11.2007 06:00:37
Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) 7 0.01% 11.11.2007 11:08:41
Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt) 6 0.01% 14.05.2005 10:43:12
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; MSIECrawler) 6 0.01% 06.10.2005 18:36:18
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322) 6 0.01% 31.01.2006 04:02:07
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; MSIECrawler) 6 0.01% 30.03.2006 07:54:43
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot) 6 0.01% 27.05.2007 02:40:29
wwwster/1.2 (Beta, mailto:[email protected]) 6 0.01% 14.10.2004 20:52:34
Scooter/3.3 6 0.01% 29.11.2005 13:02:59
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 6 0.01% 23.05.2005 04:52:19
Mozilla/5.0 (compatible; BecomeBot/1.23; +http://www.become.com/webmasters.html) 6 0.01% 31.12.2004 10:49:36
NutchCVS/0.06-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; [email protected]) 6 0.01% 30.08.2005 18:32:18
larbin_2.6.3 ([email protected]) 6 0.01% 05.07.2005 21:22:07
OmniExplorer_Bot/2.93 (+http://www.omni-explorer.com) WorldIndexer 6 0.01% 26.07.2005 17:44:05
WebIndexer/1-dev (Web Indexer; mailto://[email protected]; [email protected]) 6 0.01% 18.08.2005 21:59:24
MJ12bot/v1.0.2 (http://majestic12.co.uk/bot.php?+) 6 0.01% 02.10.2005 19:04:22
research-spider 6 0.01% 26.07.2006 09:53:09
Forschungsportal/0.8-dev (Testinstallation; http://www.forschungsportal.net/; [email protected]) 6 0.01% 12.03.2006 19:24:45
Mozilla/5.0 (X11; U; Linux i686; rv:1.8.0.4) Gecko/20060602 6 0.01% 04.07.2006 23:34:47
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; MSIECrawler) 6 0.01% 12.01.2007 14:38:00
dragonfly([email protected]) 6 0.01% 25.08.2006 21:10:21
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 6 0.01% 02.08.2006 17:03:52
Francis/2.0 ([email protected] http://www.neomo.de/pages/crawler.php) 6 0.01% 23.08.2007 11:55:23
Steeler/3.2 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 6 0.01% 10.01.2007 15:16:08
NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com) 6 0.01% 27.10.2006 04:52:25
iaskspider2 ([email protected]) 6 0.01% 07.11.2006 15:10:57
complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~networks/crawl; [email protected]) 6 0.01% 22.02.2007 20:23:35
GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html) 6 0.01% 29.09.2007 18:00:29
Pluggd/Nutch-0.9 (automated crawler; http://www.pluggd.com; support at pluggd dot com) 6 0.01% 05.10.2007 18:44:24
disco/Nutch-0.9 (experimental crawler ... please email [email protected] if problems observed; [email protected]) 6 0.01% 07.09.2007 20:37:23
Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker ([email protected]; http://www.WISEnutbot.com) 5 0.01% 06.05.2004 22:46:36
Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt) 5 0.01% 06.01.2007 23:04:27
Mozilla/4.0 (Windows XP 5.1) 5 0.01% 28.05.2005 10:25:23
LinkWalker 5 0.01% 19.05.2006 15:47:02
ObjectsSearch/0.01 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; [email protected]) 5 0.01% 02.07.2004 19:18:32
Mozilla/4.0 compatible ZyBorg/1.0 ([email protected]; http://www.WISEnutbot.com) 5 0.01% 15.06.2004 03:45:04
NutchCVS/0.05-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected]) 5 0.01% 24.08.2004 22:04:24
Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; 1u1ie4) 5 0.01% 23.08.2004 06:59:54
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; FunWebProducts; MSIECrawler) 5 0.01% 06.05.2005 19:12:20
Faxobot/1.0 5 0.01% 12.12.2004 06:03:10
oBot 5 0.01% 07.11.2005 14:30:41
Francis/2.0 ([email protected] http://www.neomo.de/) 5 0.01% 01.08.2006 08:13:12
ConveraCrawler/0.4 5 0.01% 19.01.2005 21:56:15
larbin_2.6.3 ([email protected]) 5 0.01% 16.02.2005 11:58:57
Java/1.4.2_05 5 0.01% 29.11.2005 22:59:13
Mozilla/4.0 (compatible; MSIE 5.0; Windows XP) Opera 6.01 [de] 5 0.01% 07.06.2005 23:14:18
btbot/0.4 (+http://www.btbot.com/btbot.html) 5 0.01% 15.01.2006 22:49:59
MJ12bot/v1.0.1 (http://majestic12.co.uk/bot.php?+) 5 0.01% 01.09.2005 01:37:17
NaverBot-1.0 (NHN Corp. / +82-2-3011-1989 / [email protected]) 5 0.01% 30.10.2005 02:28:34
versus crawler from [email protected] 5 0.01% 30.07.2005 15:29:29
findlinks/0.976 (+http://wortschatz.uni-leipzig.de/findlinks/) 5 0.01% 29.09.2005 17:38:41
genieBot ((http://64.5.245.11/faq/faq.html)) 5 0.01% 06.06.2006 23:23:56
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 5 0.01% 13.04.2006 11:12:30
NutchCVS/0.7.1 (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; [email protected]) 5 0.01% 30.03.2006 19:09:51
virus_detector ([email protected]) 5 0.01% 23.10.2006 21:57:44
Pico 1.0.1 (http://pico.ofdan.com) 5 0.01% 24.07.2006 18:49:51
Mozilla/5.0 (compatible; heritrix/1.6.0 +http://www.schluetersche.de) 5 0.01% 09.08.2007 03:06:27
Mozilla/5.0 (compatible;archive.org_bot/1.7.1; collectionId=316; Archive-It; +http://www.archive-it.org) 5 0.01% 17.06.2006 06:14:31
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 5 0.01% 19.07.2006 19:24:57
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; MSIECrawler) 5 0.01% 23.09.2007 16:48:16
ozelot/2.7.3 (Search engine indexer; www.flying-cat.de/ozelot; [email protected]) 5 0.01% 15.09.2006 10:22:01
Mozilla/4.0 (compatible; MSIE 6.0) 5 0.01% 09.07.2007 06:59:20
Cazoodle/Nutch-0.9-dev (Cazoodle Nutch Crawler; http://www.cazoodle.com; [email protected]) 5 0.01% 26.10.2006 05:01:44
search.ch V1.4.2 (http://www.search.ch) 5 0.01% 26.09.2007 01:27:23
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 5 0.01% 21.10.2006 14:53:05
MSRBOT (http://research.microsoft.com/research/sv/msrbot/) 5 0.01% 21.10.2006 01:38:54
Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.ancestry.com/learn/bot.aspx) 5 0.01% 17.10.2006 17:14:18
Mozilla/5.0 (compatible; MSIE 6.0; Linux) 5 0.01% 22.10.2006 17:10:36
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20061010 Firefox/2.0 5 0.01% 11.11.2006 16:59:28
Mozilla/4.0 (compatible; MSIE 6.0; Winodws NT 5.0) 5 0.01% 18.01.2007 10:23:29
Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea) 5 0.01% 21.03.2007 04:28:25
Wget/1.10.2 5 0.01% 13.10.2007 03:23:27
Mozilla/5.0 (compatible; Exabot-Images/3.0; +http://www.exabot.com/go/robot) 5 0.01% 09.04.2007 10:22:44
Baiduspider+(+http://www.baidu.com/search/spider_jp.html) 5 0.01% 24.07.2007 23:00:59
lwp-trivial/1.41 5 0.01% 21.04.2007 16:59:46
Mozilla/5.0 (compatible; SkreemRBot +http://skreemr.com) 5 0.01% 04.10.2007 00:53:58
thumbshots-de-Bot (Version: 1.11, powered by www.thumbshots.de) 5 0.01% 02.10.2007 20:07:50
Gigabot/2.0att 5 0.01% 21.07.2007 17:05:28
Sirketcebot/v.01 (http://www.sirketce.com/bot.html) 5 0.01% 13.07.2007 14:26:52
kalooga/Nutch-0.9 5 0.01% 19.09.2007 03:31:57
AOL_Ian_The_Aussie_Spider/Nutch-0.9 (Demo Thing; [email protected]) 5 0.01% 11.11.2007 08:01:39
DjangoTraineeBot/0.2 5 0.01% 20.10.2007 11:47:24
FAST-WebCrawler/3.8 (crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler) 4 0.01% 13.04.2004 08:10:29
Openfind data gatherer, Openbot/3.0+([email protected];+http://www.openfind.com.t w/robot.html) 4 0.01% 20.07.2005 22:08:56
NPBot (http://www.nameprotect.com/botinfo.html) 4 0.01% 25.11.2004 17:16:08
Vagabondo/2.0 MT (webagent at wise-guys dot nl) 4 0.01% 05.01.2005 21:32:27
Steeler/2.0 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 4 0.01% 03.06.2004 07:30:13
Mozilla/4.0 (compatible; Synapse) 4 0.01% 20.06.2005 20:20:48
Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.0 (XDF) 4 0.01% 07.08.2004 20:37:11
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98) 4 0.01% 28.03.2005 12:59:55
Mozilla/5.0 (Slurp/si; [email protected]; http://www.inktomi.com/slurp.html) 4 0.01% 14.09.2004 11:39:45
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.4.1) Gecko/20020508 Netscape6/6.2.3 4 0.01% 24.05.2005 21:35:29
CreativeCommons/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected]) 4 0.01% 07.10.2005 08:13:26
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.5) Gecko/20041108 Firefox/1.0 4 0.01% 31.03.2005 08:52:08
Mozilla/4.0 (compatible; grub-client-2.3) 4 0.01% 20.01.2005 09:24:06
http://www.istarthere.com ([email protected]) 4 0.01% 30.01.2005 21:20:45
thesubot-beta-www.thesubot.de 4 0.01% 14.02.2005 03:06:27
Mozilla/5.0 (Windows XP; U) Opera 6.01 [en]" 4 0.01% 11.05.2005 11:47:43
versus crawler [email protected] 4 0.01% 28.05.2005 10:18:07
Steeler/3.1 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 4 0.01% 26.04.2006 19:05:04
findlinks/0.903 (+http://wortschatz.uni-leipzig.de/findlinks/) 4 0.01% 02.05.2005 07:21:34
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 4 0.01% 07.05.2005 16:34:45
Mozilla 4 0.01% 29.08.2006 13:25:56
Mozilla/5.0 (OS/2; U; Warp 4.5; en-US; rv:0.9.8) Gecko/20020204 4 0.01% 28.05.2005 10:49:20
Mozilla/4.61 [en] (OS/2; U) 4 0.01% 04.06.2005 03:14:58
Mozilla/4.75 [en] (X11; U; SunOS 5.8 sun4u) 4 0.01% 17.06.2005 02:56:40
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; .NET CLR 1.1.4322; MSIECrawler) 4 0.01% 01.11.2005 14:29:53
unido-bot, http://mobicom.cs.uni-dortmund.de/bot.html 4 0.01% 21.02.2006 23:34:47
INGRID/3.0 MT ([email protected]; http://webmaster.ilse.nl/jsp/webmaster.jsp) 4 0.01% 07.10.2005 06:39:52
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) 1Noonbot 1.0 mailto:[email protected] 4 0.01% 30.07.2005 18:17:58
OmniExplorer_Bot/2.35 (+http://www.omni-explorer.com) WorldIndexer 4 0.01% 13.07.2005 18:43:00
Omnipelagos.com RSS Spider 0.3 4 0.01% 25.08.2005 06:42:28
findlinks/0.966 (+http://wortschatz.uni-leipzig.de/findlinks/) 4 0.01% 15.09.2005 19:42:17
Mozilla/4.0 (compatible; GPU p2p crawler http://gpu.sourceforge.net/search_engine.php) 4 0.01% 29.10.2005 12:39:32
stalker/0.1 (http://www.stalkpire.org/bot.htm) 4 0.01% 23.08.2006 14:11:01
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Telekom Austria; FunWebProducts; Hotbar 4.5.1.0; MSIECrawler) 4 0.01% 05.07.2007 19:31:13
findlinks/1.0 (+http://wortschatz.uni-leipzig.de/findlinks/) 4 0.01% 05.11.2005 16:53:24
Mozdex/0.7.2-dev (Mozdex; http://www.mozdex.com/bot.html; [email protected]) 4 0.01% 15.02.2006 23:36:16
IlTrovatore-Setaccio/1.2 (It search engine; http://www.iltrovatore.it/bot.html; [email protected]) 4 0.01% 02.11.2005 17:19:27
Shim-Crawler 4 0.01% 14.11.2005 00:16:09
yacy (www.yacy.net; v20040602; i386 Linux 2.4.20-021stab028.17.777-enterprise; java 1.4.2_08; Europe/en) 4 0.01% 06.02.2006 13:01:22
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 4 0.01% 22.09.2007 09:53:22
http://www.almaden.ibm.com/cs/crawler 4 0.01% 23.06.2006 14:20:09
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Firefox/1.5 4 0.01% 31.01.2006 20:32:10
http://pressemitteilung.ws/ 4 0.01% 29.12.2006 09:10:42
Mozilla/5.0 (compatible; Theophrastus/2.1; +http://users.cs.cf.ac.uk/N.A.Smith/theophrastus.php) 4 0.01% 06.01.2006 18:01:13
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705) 4 0.01% 28.01.2006 06:42:07
FAST Enterprise Crawler/6.4 ([email protected]) 4 0.01% 24.03.2006 04:13:56
sproose/0.1-alpha (sproose crawler; http://www.sproose.com/bot.html; [email protected]) 4 0.01% 01.04.2006 19:45:00
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_06; Europe/de) 4 0.01% 23.07.2006 19:30:33
ccubee/3.5 4 0.01% 05.10.2006 16:06:35
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.1) Gecko/20060124 Firefox/1.5.0.1 4 0.01% 27.07.2006 12:09:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Naver Desktop Search) 4 0.01% 07.07.2006 02:26:50
Yeti 4 0.01% 08.06.2006 05:40:42
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5 4 0.01% 13.09.2006 15:26:31
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 4 0.01% 31.10.2006 20:07:45
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5) Gecko 4 0.01% 25.10.2006 16:28:27
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 2.0.50727; .NET CLR 1.1.4322; Media Center PC 4.0; FDM; Media Center PC 3.0; MSIECrawler) 4 0.01% 01.01.2007 20:15:13
Mozilla/5.0 ([email protected]) 4 0.01% 06.01.2007 23:23:44
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; MSIECrawler) 4 0.01% 26.09.2007 16:55:04
ConveraCrawler/0.9e (+http://www.authoritativeweb.com/crawl) 4 0.01% 15.05.2007 18:13:21
Mozilla/5.0 (compatible; Najdi.si/3.1) 4 0.01% 26.03.2007 12:08:11
LarbinWebCrawler ([email protected]) 4 0.01% 18.03.2007 10:39:01
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705) 4 0.01% 24.05.2007 18:17:57
ICCrawler - ICjobs (http://www.icjobs.de/bot.htm) 4 0.01% 15.09.2007 04:19:39
Twiceler-0.9 http://www.cuill.com/twiceler/robot.html 4 0.01% 23.04.2007 09:40:25
Speedy Spider (Entireweb; Beta/1.2; http://www.entireweb.com/about/search_tech/speedyspider/) 4 0.01% 09.05.2007 22:51:56
ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl-contact(at)ml(dot)nict(dot)go(dot)jp) 4 0.01% 24.06.2007 03:44:52
Nutch/Nutch-0.8.1 (Nutch; Nutch; Nutch) 4 0.01% 03.07.2007 04:00:52
Snapbot/1.0 (Snap Shots, +http://www.snap.com) 4 0.01% 24.09.2007 00:19:32
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://www.changedetection.com/bot.html ) 4 0.01% 26.07.2007 00:22:52
terraminds-bot/1.0 ([email protected]) 4 0.01% 14.08.2007 16:16:46
larbin_2.6.3 4 0.01% 15.09.2007 07:28:13
I411Crawling/Nutch-0.9 (CrawlData; www.i411.com; [email protected]) 4 0.01% 17.10.2007 15:05:32
Toplistbot 4 0.01% 08.11.2007 17:28:48
mozDex/0.04-dev (mozDex; http://www.mozdex.com/bot.html; [email protected]) 3 0.01% 21.06.2004 11:08:40
NutchCVS/0.03-dev (Nutch; http://www.nutch.org/docs/en/bot.html; [email protected]) 3 0.01% 27.10.2005 11:50:35
Iltrovatore-Setaccio/0.3-dev (Indexing; http://www.iltrovatore.it/bot.html; [email protected]) 3 0.01% 15.04.2004 12:51:06
WebFilter Robot 1.0 3 0.01% 29.09.2004 19:14:45
Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.6) Gecko/20040210 Firefox/0.8 3 0.01% 27.04.2004 22:46:00
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; MSIECrawler) 3 0.01% 23.05.2006 15:01:31
IlTrovatore-Setaccio/1.2 (Indexing; http://www.iltrovatore.it/bot.html; [email protected]) 3 0.01% 16.07.2004 12:53:03
GeonaBot/1.1; http://www.geona.com/ 3 0.01% 04.03.2005 13:06:23
boitho.com-dc/0.51 ( http://www.boitho.com/dcbot.html ) 3 0.01% 22.07.2004 12:35:14
ObjectsSearch/0.05 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; [email protected]) 3 0.01% 04.08.2004 07:51:55
Java/1.4.2_04 3 0.01% 28.01.2005 23:42:36
Mozilla/5.0 3 0.01% 28.08.2007 20:12:49
pipeLiner/0.3a (PipeLine Spider; http://www.pipeline-search.com/webmaster.html; [email protected]) 3 0.01% 11.01.2005 19:05:38
Wget/1.8.2 3 0.01% 07.08.2005 17:10:58
Yahoo-MMAudVid/1.0 (mms dash mmaudvidcrawler dash support at yahoo dash inc dot com) 3 0.01% 24.02.2005 04:54:43
Digger/2.0 JDK/1.4.2 3 0.01% 08.05.2006 23:06:58
Mozilla/5.0 (compatible; Yahoo! Slurp/si-emb; http://help.yahoo.com/help/us/ysearch/slurp) 3 0.01% 23.05.2006 10:43:29
NP/0.1 (NP; http://www.nameprotect.com; [email protected]) 3 0.01% 19.03.2005 12:24:05
Mozilla/4.0 (compatible; SpeedySpider; www.entireweb.com) 3 0.01% 27.03.2005 16:55:54
Mozilla/5.0 (compatible; BecomeBot/1.83; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 3 0.01% 26.04.2005 02:10:14
Mozdex/0.06-dev (Mozdex; http://www.mozdex.com/bot.html; [email protected]) 3 0.01% 25.04.2005 11:27:00
MicroBaz 3 0.01% 16.06.2005 19:35:35
DataparkSearch/4.30-04052005 (+http://www.dataparksearch.org/) 3 0.01% 20.05.2005 11:58:19
pipeLiner/0.12 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html) 3 0.01% 21.07.2005 08:47:14
W3C-checklink/4.2 [4.20] libwww-perl/5.803 3 0.01% 30.07.2005 19:16:59
Mozilla/5.0 (compatible; BecomeBot/1.86; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 3 0.01% 24.05.2005 18:08:07
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT) 3 0.01% 14.06.2005 17:07:13
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0) 3 0.01% 17.08.2005 09:07:57
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) Accept: */* Accept-Language: en-us, en, de;q=0.50 3 0.01% 04.06.2005 09:18:20
NutchCVS/0.70-dev-mnebel-20050219 (Netluchs; http://www.netluchs.de/; [email protected]) 3 0.01% 02.07.2005 02:23:31
ccubee/3.0 3 0.01% 05.07.2005 12:19:46
Seekbot/2.x 3 0.01% 05.09.2005 02:06:27
Onet.pl SA, http://szukaj.onet.pl 3 0.01% 24.07.2005 09:13:25
sna-0.0.1 ([email protected]) 3 0.01% 12.07.2006 22:34:45
ccubee/3.1 3 0.01% 15.07.2005 01:45:01
libwww-perl/5.803 3 0.01% 25.11.2005 15:31:25
Searchmee! Spider v0.98a 3 0.01% 26.04.2006 20:07:50
unknown ([email protected]) 3 0.01% 25.08.2005 23:02:03
ZeBot_www.ze.bz ([email protected]) 3 0.01% 15.10.2005 12:54:59
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.10) Gecko/20050717 Firefox/1.0.6 3 0.01% 23.09.2005 17:41:56
iCCrawler (http://www.iccenter.net) 3 0.01% 15.10.2005 23:05:36
NG-Search/0.85_(+http://www.ng-search.com) 3 0.01% 30.09.2005 11:57:03
Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece) 3 0.01% 08.10.2006 10:29:43
Metaspinner/0.01 (Metaspinner; http://www.meta-spinner.de/; [email protected]/) 3 0.01% 30.10.2005 07:47:17
Shim Crawler 3 0.01% 23.10.2005 19:55:59
ccubee/3.2 3 0.01% 21.11.2005 23:01:40
Blogpulse ([email protected]) 3 0.01% 30.10.2005 16:11:50
RufusBot (Rufus Web Miner; http://64.124.122.252/feedback.html) 3 0.01% 01.11.2005 00:14:09
Java/1.5.0_04 3 0.01% 23.05.2006 08:21:25
gonzo1[D] mailto:[email protected] 3 0.01% 22.06.2006 19:13:08
findlinks/1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/) 3 0.01% 17.12.2005 01:00:02
findlinks/1.1-a4 (+http://wortschatz.uni-leipzig.de/findlinks/) 3 0.01% 05.01.2006 01:12:25
MJ12bot/v1.0.6 (http://majestic12.co.uk/bot.php?+) 3 0.01% 23.12.2005 16:37:23
Alpha Search Agent 3 0.01% 10.01.2006 20:00:56
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; freenet.de frndial 5.0; MSIECrawler) 3 0.01% 26.01.2006 14:00:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; MSIECrawler) 3 0.01% 28.10.2006 10:59:56
Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) 3 0.01% 17.03.2006 23:00:05
yacy (www.yacy.net; v20040602; i386 Linux 2.6.8-022stab070.4-enterprise; java 1.5.0_06; Europe/en) 3 0.01% 29.04.2006 22:19:30
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (R1 1.5); .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 3.1; Media Center PC 3.0) 3 0.01% 24.04.2006 04:05:56
Randy 3 0.01% 27.05.2006 01:26:28
topicblogs/0.9 3 0.01% 30.06.2007 20:19:04
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 3 0.01% 24.06.2006 20:49:44
updated/0.1-alpha (updated crawler; http://www.updated.com; [email protected]) 3 0.01% 08.06.2006 19:15:04
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Alcohol Search; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MSIECrawler) 3 0.01% 11.06.2006 19:51:46
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Wysigot 5.51; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MSIECrawler) 3 0.01% 23.06.2006 12:44:12
W3C-checklink/4.2.1 [4.21] libwww-perl/5.805 3 0.01% 01.07.2006 14:09:23
BeijingCrawler 3 0.01% 28.07.2006 03:56:39
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1; MSIECrawler) 3 0.01% 11.08.2007 21:18:09
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 3 0.01% 02.08.2006 08:50:53
yacy (www.yacy.net; v20040602; i386 Linux 2.6.13-15.11-default; java 1.5.0_07; Europe/en) 3 0.01% 04.02.2007 13:01:19
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 3 0.01% 18.08.2006 09:30:51
Mozilla/5.0 (Windows; U; Windows NT 5.2; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 3 0.01% 27.08.2006 14:35:11
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 3 0.01% 17.09.2006 17:41:22
Mozilla/5.0 (Windows;) NimbleCrawler 2.0.1 obeys UserAgent NimbleCrawler For problems contact: [email protected] 3 0.01% 18.07.2007 09:25:35
NG-Search/0.9.8 (NG-SearchBot; http://www.ng-search.com) 3 0.01% 28.09.2006 21:59:33
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; FDM; MSIECrawler) 3 0.01% 10.10.2006 02:16:28
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 Mnenhy/0.7.4.666 3 0.01% 10.10.2006 00:47:24
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20060918 Firefox/2.0 3 0.01% 05.10.2006 00:00:40
gonzo1[P] +http://www.suchen.de/popups/faq.jsp 3 0.01% 07.10.2006 19:27:17
silk/1.0 3 0.01% 26.10.2006 00:52:55
DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html) 3 0.01% 07.03.2007 16:40:48
hohotestbot/Nutch-0.8.1 (HOHO Test Bot (Nutch); no.urlyetjustesting.com) 3 0.01% 17.03.2007 21:21:17
Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete_main.asp) 3 0.01% 10.11.2007 17:41:49
ParaginSpider/Nutch-0.8.1 3 0.01% 03.02.2007 01:05:24
Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; [email protected]) 3 0.01% 02.02.2007 13:16:59
OpenISearch/1.x (www.openisearch.com) 3 0.01% 26.01.2007 03:08:02
Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 5.0 Robot) 3 0.01% 05.04.2007 18:48:14
boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html ) 3 0.01% 09.11.2007 04:33:26
Mozilla/5.0 (compatible; OsO; http://oso.octopodus.com/abot.html) 3 0.01% 13.10.2007 21:49:14
StreamSzene Spider/Nutch-0.9 (Spider from StreamSzene.de; [email protected]) 3 0.01% 10.05.2007 00:56:36
nokia6610I/1.0 (4.10) Profile/MIDP-1.0 Configuration/CLDC-1.0 (FAST WAP Proxy/1.0) 3 0.01% 21.09.2007 17:37:12
great-plains-web-spider/gpws (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected]) 3 0.01% 01.07.2007 10:12:09
ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; [email protected]) 3 0.01% 12.06.2007 07:15:04
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) 3 0.01% 25.09.2007 13:15:40
VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; [email protected]) 3 0.01% 12.07.2007 19:13:20
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; +http://process1.com) Gecko/20070508 Firefox/1.5.0.12 3 0.01% 06.09.2007 11:56:57
Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) ) 3 0.01% 17.08.2007 17:30:14
TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net) 3 0.01% 02.09.2007 12:24:51
visit_www.realfrags.com ([email protected]) 3 0.01% 25.08.2007 07:06:33
http://pressemitteilung.ws/spamfilter 3 0.01% 03.09.2007 13:23:41
disco/Nutch-0.9 (experimental crawler ... please email [email protected] if problems observed; [email protected]) 3 0.01% 12.09.2007 17:58:49
disco/Nutch-0.9 (experimental crawler; [email protected]) 3 0.01% 22.09.2007 14:15:07
Mozilla/4.0(compatible;MSIE5.00;Windows98) ([email protected]) 3 0.01% 26.10.2007 06:39:03
Mozilla/5.0 (compatible; heritrix/1.12.0 +http://seekda.com) 3 0.01% 09.11.2007 13:41:45
DjangoTraineeBot/0.1 3 0.01% 17.10.2007 08:17:29
Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+) 3 0.01% 12.11.2007 01:25:22
Googlebot/Test (+http://www.googlebot.com/bot.html) 2 0% 20.04.2004 04:25:00
Tutorial Crawler 1.4 (http://www.tutorgig.com/crawler) 2 0% 21.05.2004 16:36:34
IlTrovatore-Setaccio/1.2-dev (Indexing; http://www.iltrovatore.it/bot.html; [email protected]) 2 0% 08.05.2004 11:59:57
IlTrovatore-Setaccio/1.2 (Indexing; http://www.iltrovatore.it/bot.html; [email protected]) 2 0% 15.06.2004 18:12:14
larbin_2.6.3_for_(http://cosco.hiit.fi/search/) ([email protected]) 2 0% 11.07.2004 01:57:41
mozDex/0.05-dev (mozDex; http://www.mozdex.com/bot.html; [email protected]) 2 0% 02.07.2004 04:47:53
Wget/1.9.1 2 0% 24.05.2005 10:48:04
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 2 0% 03.05.2005 15:03:53
augurfind V-1.8 2 0% 15.08.2004 22:08:33
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; IBP) 2 0% 20.09.2004 09:15:17
Scooter/3.3Y!CrawlX 2 0% 03.09.2004 14:59:56
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM; MSIECrawler) 2 0% 10.09.2004 17:18:41
ZipppBot/0.11 (ZipppBot; http://www.zippp.net; [email protected]) 2 0% 24.10.2004 23:36:39
NextGenSearchBot 1 (for information visit http://www.eliyon.com/NextGenSearchBot) 2 0% 06.05.2005 02:45:49
Mozilla/4.0 (stat 0.11) ([email protected]) 2 0% 30.10.2004 23:36:47
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0) 2 0% 17.10.2004 11:48:21
W3C-checklink/4.0 [4.4] libwww-perl/5.800 2 0% 26.10.2004 14:36:33
Mozilla/4.7 [en]([email protected]) 2 0% 21.11.2004 17:32:24
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Opera 7.54 [de] 2 0% 26.11.2004 07:39:17
MJ12bot/v0.5.0 (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php) run by PeerID=E71E149BF7B07EFCFE510E729DD2B9E3 MemberID=C9403BB515387FBB631AC512950E6F0E 2 0% 26.11.2004 19:07:36
getRAX/getRAX Crawler 07.1 (+http://www.getRAX.com) 2 0% 30.11.2004 09:03:44
Motoricerca-Robots.txt-Checker/1.0 (http://tool.motoricerca.info/robots-checker.phtml) 2 0% 30.11.2004 23:27:46
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (R1 1.5); MSIECrawler) 2 0% 03.12.2004 19:49:24
gazz/5.0 ([email protected]) 2 0% 24.12.2004 21:56:32
Java/1.4.1_02 2 0% 12.05.2005 08:58:00
MJ12bot/v0.8.5 (http://www.majestic12.co.uk/projects/dsearch/mj12bot.php?V=v0.8.5&NID =00B05BD11311C153&MID=7A764EA9585B2914&BID=C6B394EFB2CEBC23A5467AD31F8 1D2B9 2 0% 27.01.2005 04:59:47
Mozilla/1.0 2 0% 31.03.2005 22:08:02
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.0.0) Gecko/20020530 2 0% 19.02.2005 17:48:44
Mozilla/5.0 (compatible; BecomeBot/2.2.1; MSIE 6.0 compatible; +http://www.become.com/webmasters.html) 2 0% 24.02.2005 03:25:31
eventax/1.3 (eventax; http://www.eventax.de/; [email protected]) 2 0% 21.06.2005 02:37:50
pipeLiner/0.10 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html) 2 0% 13.03.2005 06:11:26
PhpDig/1.8.7 (+http://www.phpdig.net/robot.php) 2 0% 22.02.2005 23:42:50
Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 FAKE (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 2 0% 03.03.2005 23:42:00
thumbshots-de-Bot (Version: 1.02, powered by www.thumbshots.de) 2 0% 24.08.2006 19:23:30
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de-DE; rv:1.7.5) Gecko/20041108 Firefox/1.0 StumbleUpon/1.999 2 0% 15.03.2005 22:15:45
ObjectsSearch/0.07 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; [email protected]) 2 0% 19.03.2005 02:16:22
MJ12bot/v0.9.0 (http://majestic12.co.uk/bot.php?+) 2 0% 20.03.2005 19:55:51
BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html) 2 0% 19.06.2005 23:00:22
Steeler/3.0 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 2 0% 11.04.2005 21:24:14
Mozilla/5.0 (compatible; BecomeBot/2.2.5; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 2 0% 14.05.2005 01:12:42
Mozilla/5.0 ([email protected]) 2 0% 27.04.2005 05:37:30
AIBOT/2.1 By +(www.21seek.com , A Real artificial intelligence search engine , China) 2 0% 24.07.2005 19:34:36
UnChaos, From Chaos To Order, Hybrid Web Search Engine at http://www.unchaos.com ([email protected]) 2 0% 07.06.2005 21:59:42
NutchCVS/0.06-dev (WebCam search project-U of Washington ; http://www.nutch.org/docs/en/bot.html; [email protected]) 2 0% 03.06.2005 04:31:44
Mozilla/4.76 [en] (WinNT; U) 2 0% 08.06.2005 00:51:14
EuripBot/0.2 (+http://www.eurip.com) GetRobots 2 0% 17.08.2005 11:25:14
Mozilla/4.0 (SunOS 5.8) 2 0% 14.06.2005 17:23:27
HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp) 2 0% 28.09.2006 03:40:34
NCSA Beta 1 (http://vias.ncsa.uiuc.edu/viasarchivinginformation.html) 2 0% 21.06.2005 03:09:14
findlinks/0.939 (+http://wortschatz.uni-leipzig.de/findlinks/) 2 0% 13.07.2005 21:17:12
mozilla/5.0 (compatible; heritrix/1.2.0 +http://www.offis.de) 2 0% 13.07.2005 10:04:00
WebIndexer/1-dev (WebIndexer; [email protected]; [email protected]) 2 0% 14.07.2005 15:55:27
OmniExplorer_Bot/2.57 (+http://www.omni-explorer.com) WorldIndexer 2 0% 15.07.2005 18:48:41
OmniExplorer_Bot/2.82 (+http://www.omni-explorer.com) WorldIndexer 2 0% 20.07.2005 22:28:46
UofTDB_experiment ([email protected]) 2 0% 23.07.2005 01:56:05
ConveraCrawler/0.9 (+http://www.authoritativeweb.com/crawl) 2 0% 01.08.2005 02:35:46
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 3.1; MSIECrawler) 2 0% 10.01.2007 19:15:06
findlinks/0.956 (+http://wortschatz.uni-leipzig.de/findlinks/) 2 0% 30.08.2005 05:01:44
8pms 2 0% 16.08.2005 13:45:42
t 2 0% 16.08.2005 21:10:04
Gp85t 2 0% 17.08.2005 15:10:31
Mozilla/4.0 (Windows NT 4.0) 2 0% 23.08.2005 18:16:14
Virgo/0.7 (-; http://virgo.tiedonhaku.fi/; n/a) 2 0% 30.08.2005 07:13:18
IconSurf/2.0 favicon finder (see http://iconsurf.com/robot.html) 2 0% 06.09.2005 09:51:18
eagle Accept: */* Referer: http://www.licats.com/ 2 0% 07.09.2005 15:26:27
OmniWeb 2 0% 02.04.2007 20:04:44
findlinks/0.968 (+http://wortschatz.uni-leipzig.de/findlinks/) 2 0% 15.09.2005 14:41:22
SBIder/0.7 (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html) 2 0% 21.09.2005 09:04:26
NutchOSU-VLIB/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 2 0% 07.10.2005 21:57:46
findlinks/0.973 (+http://wortschatz.uni-leipzig.de/findlinks/) 2 0% 20.09.2005 20:05:29
Teoma 2 0% 04.11.2005 05:27:25
FAST Enterprise Crawler 6 used by Sensis.com.au Web Crawler (search_commentsatsensisdotcomdotau) 2 0% 30.09.2005 16:21:57
InternetArchive/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 2 0% 28.09.2005 06:20:38
Wget/1.10.1 2 0% 13.03.2006 10:18:12
CsCrawler RPT-HTTPClient/0.3-3 2 0% 20.10.2005 21:45:20
DoCoMo/2.0 P900i(c100;TB;W24H11) 2 0% 14.10.2005 20:16:04
KFSW-Bot (Version: 1.01, powered by KFSW, www.kfsw.de) 2 0% 26.10.2005 19:39:49
Deepindex 2 0% 21.10.2005 04:26:46
IRLbot/2.0 (+http://irl.cs.tamu.edu/crawler) 2 0% 26.01.2006 04:34:01
augurfind V-2 beta1 2 0% 16.04.2006 17:41:24
findlinks/1.01 (+http://wortschatz.uni-leipzig.de/findlinks/) 2 0% 28.10.2005 02:40:05
MJ12bot/v1.0.4 (http://majestic12.co.uk/bot.php?+) 2 0% 25.11.2005 10:40:27
OutfoxBot/0.3 (For internet experiments; http://; [email protected]) 2 0% 12.11.2005 22:34:33
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; HbTools 4.7.2; MSIECrawler) 2 0% 18.11.2005 20:03:44
e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/) 2 0% 29.12.2005 18:30:31
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; PeoplePal 6.2; MSIECrawler) 2 0% 22.11.2005 03:46:18
ejupiter.com 2 0% 26.11.2005 11:06:44
Mozilla/5.0 (compatible; Kyluka krawl; http://www.kyluka.com/krawl.html) 2 0% 04.12.2005 02:57:50
SynooBot/0.7.1 (SynooBot; http://www.synoo.de/bot.html; [email protected]) 2 0% 13.12.2005 22:27:56
Mozilla/5.0 (compatible; OnetSzukaj/5.0; +http://szukaj.onet.pl 2 0% 08.01.2006 07:46:55
VORTEX/1.2 (+http://marty.anstey.ca/robots/vortex/) 2 0% 27.12.2005 18:42:22
MapoftheInternet.com (+http://MapoftheInternet.com) 2 0% 26.01.2006 12:05:35
ccubee/4.0 2 0% 01.02.2006 09:46:15
yacy (www.yacy.net; v20040602; ppc Mac OS X 10.4.4; java 1.4.2_09; Europe/de) 2 0% 09.02.2006 00:39:40
Forschungsportal/0.8-dev (Testinstallation; http://www.forschungsportal.net/; [email protected]) 2 0% 05.02.2006 13:57:35
FAST Enterprise Crawler 6 / Scirus [email protected]; http://www.scirus.com/srsapp/contactus/ 2 0% 04.02.2006 14:59:23
PhpDig/1.8.9 RC1 (+http://www.phpdig.net/robot.php) 2 0% 14.02.2006 06:35:38
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7 2 0% 02.03.2006 02:12:44
Mozilla/5.0 (Windows;) NimbleCrawler 1.15 obeys UserAgent NimbleCrawler For problems contact: [email protected] 2 0% 05.03.2006 03:16:38
NuSearch Spider (compatible; MSIE 6.0) 2 0% 04.03.2006 09:59:50
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 2 0% 11.03.2006 22:19:29
CSCrawler -> http://www.kde.cs.uni-kassel.de/lehre/ss2005/googlespam/crawler.html RPT-HTTPClient/0.3-3 2 0% 28.03.2006 01:46:33
LinkChecker/3.4 (+http://linkchecker.sourceforge.net/) 2 0% 25.04.2006 15:19:01
OmniExplorer_Bot/6.47 (+http://www.omni-explorer.com) WorldIndexer 2 0% 18.03.2006 11:27:50
Mozilla/5.0 (compatible; vermut +http://vermut.aol.com) 2 0% 31.05.2006 12:53:09
Mozilla/5.0 (Windows;) NimbleCrawler 2.0.0 obeys UserAgent NimbleCrawler For problems contact: [email protected] 2 0% 15.04.2006 03:15:27
larbin_2.6.3 ([email protected]) 2 0% 06.05.2006 06:04:53
Privoxy/3.0 (Anonymous) 2 0% 18.04.2006 17:59:42
Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 2 0% 29.04.2006 19:09:31
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 2 0% 22.05.2006 19:52:48
PeerFactor Crawler 2 0% 10.05.2006 15:37:05
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 2 0% 26.05.2006 14:13:26
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 2 0% 24.05.2006 14:23:13
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 2 0% 01.06.2006 06:11:41
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 2 0% 09.06.2006 15:51:12
Swooglebot/2.0. (+http://swoogle.umbc.edu/swooglebot.html) 2 0% 10.04.2007 04:13:05
Blaiz-Bee/2.00.5502 (+http://www.blaiz.net) 2 0% 24.07.2006 22:17:21
Gigabot 2 0% 10.02.2007 05:13:16
FAST Enterprise Crawler 6 used by WWU ([email protected]) 2 0% 29.06.2006 08:52:42
exactseek.com 2 0% 30.06.2006 15:55:29
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 1&1 Internet AG by USt; .NET CLR 1.0.3705; MSIECrawler) 2 0% 14.07.2006 14:58:02
MetagerBot/0.8-dev (MetagerBot; http://metager.de; ) 2 0% 22.07.2006 22:36:49
Szukacz/1.5 (robot; www.szukacz.pl/html/jak_dziala_robot.html; [email protected]) 2 0% 28.08.2006 10:44:46
Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.8) Gecko/20051224 Debian/1.5.dfsg-3 Firefox/1.5 2 0% 21.08.2006 11:41:13
IlseBot/1.0 2 0% 05.10.2006 05:10:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) () 2 0% 18.09.2006 23:00:12
Mozilla/4.0 (compatible; NaverBot/1.0; [email protected]) 2 0% 06.09.2006 22:54:20
kakle-spider/0.1 (kakle-spider; http://www.kakle.com/bot.html; [email protected]) 2 0% 10.09.2006 23:36:41
YooW!/2.0.0 (+http://www.yoow.eu) 2 0% 14.09.2006 05:25:18
libwww-perl/5.800 2 0% 26.05.2007 18:20:46
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_08; Europe/de) 2 0% 21.09.2006 03:20:04
Mozilla/5.0 (Linux; en-US; Netscape 7) 2 0% 15.10.2006 16:47:09
Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.0; DAUM Web Robot; Daum Communications Corp., Korea) 2 0% 25.12.2006 01:22:05
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1) Gecko/20060601 Firefox/2.0 (Ubuntu-edgy) 2 0% 23.12.2006 15:24:57
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.6.0; Europe/de) 2 0% 17.01.2007 15:18:50
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_10; Europe/de) 2 0% 26.01.2007 12:45:56
eApolloBot/1.0 (eApollo search engine robot; http://www.eapollo.com; eapollo at global-opto dot com) 2 0% 29.01.2007 11:20:16
NetWhatCrawler/0.06-dev (NetWhatCrawler from NetWhat.com; http://www.netwhat.com; [email protected]) 2 0% 14.07.2007 09:25:10
yacy (www.yacy.net; v20040602; i386 Linux 2.6.18; java 1.5.0_09; Europe/de) 2 0% 14.02.2007 15:47:05
yacy (www.yacy.net; v20040602; amd64 Linux 2.6.16-2-amd64-k8-smp; java 1.5.0_08; Europe/en) 2 0% 01.04.2007 22:25:35
Mozilla 4.3 2 0% 01.03.2007 00:51:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; MSIECrawler) 2 0% 18.03.2007 14:43:05
Touche 2 0% 28.06.2007 18:20:58
(null) 2 0% 21.03.2007 11:05:26
Webscope/Nutch-0.9-dev (http://www.cs.washington.edu/homes/mjc/agent.html) 2 0% 24.03.2007 09:59:12
crawler/10.36 2 0% 13.04.2007 18:13:22
sogou web spider(+http://www.sogou.com/docs/help/webmasters.htm#07) 2 0% 02.04.2007 19:28:53
SynooBot/SynooBot (compatible; Synoobot/0.9; http://www.synoo.com/search/bot.html) 2 0% 02.04.2007 09:38:31
boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html ) 2 0% 23.05.2007 18:39:03
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; Media Center PC 4.0; MSIECrawler) 2 0% 01.04.2007 18:26:25
NutchSIS/Nutch-0.8.1 2 0% 08.04.2007 10:46:54
Java/1.6.0 2 0% 27.04.2007 02:18:58
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; eBook; MSIECrawler) 2 0% 02.05.2007 15:00:45
Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/) 2 0% 05.11.2007 18:17:21
Yeti/0.01 (nhn/1noon, [email protected], check robots.txt daily and follows it) 2 0% 09.07.2007 15:35:56
alef/0.0 2 0% 11.05.2007 05:58:06
yacy (www.yacy.net; v20040602; ppc Mac OS X 10.4.9; java 1.5.0_07; Europe/de) 2 0% 14.05.2007 21:52:52
Nutch/Nutch-0.9 2 0% 22.05.2007 07:40:04
Mozilla/4.0 (Windows XP 5.1) Java/1.5.0_09 2 0% 22.05.2007 18:43:48
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.6.0_01; Europe/de) 2 0% 29.06.2007 02:06:34
Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com) 2 0% 01.07.2007 04:44:56
Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; [email protected]) 2 0% 19.06.2007 21:04:14
Mail.Ru/1.0 2 0% 18.09.2007 11:31:35
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.2; MSIECrawler) 2 0% 14.06.2007 19:00:32
Gallent Search Spider v1.4 Robot 3 (http://robot.GallentSearch.com) 2 0% 15.06.2007 03:12:28
yacy (www.yacy.net; v20040602; i386 Linux 2.6.20.1; java 1.4.2_13; Europe/en) 2 0% 16.07.2007 20:59:21
yacy (www.yacy.net; v20040602; x86 Windows 2003 5.2; java 1.6.0_01; Europe/de) 2 0% 03.07.2007 13:04:40
Mozilla/5.0 (compatible; archive.org_bot/1.13.1x +http://crawler.archive.org) 2 0% 26.07.2007 08:32:42
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; SIMBAR={C21297C2-7B18-40ad-A7BE-FB0C6FD61D1F}; .NET CLR 2.0.50727; .NET CLR 1.1.4322; MSIECrawler) 2 0% 06.09.2007 10:38:41
multicrawler (+http://sw.deri.org/2006/04/multicrawler/robots.html) 2 0% 22.08.2007 18:12:36
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.6.0_02; Europe/de) 2 0% 20.09.2007 02:45:23
Pagebull http://www.pagebull.com/ 2 0% 17.10.2007 05:02:47
Mozilla/5.0 (compatible; Bot; +http://pressemitteilung.ws/spamfilter 2 0% 11.10.2007 18:03:07
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 2 0% 01.10.2007 00:13:33
Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6 2 0% 18.09.2007 06:22:35
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 ([email protected]) 2 0% 20.09.2007 12:27:54
bubblenutch/Nutch-0.9 (www.bubblecast.ru bot; www.bubblecast.ru; [email protected]) 2 0% 26.10.2007 04:57:49
Pete-Spider Light/1.0 2 0% 03.10.2007 14:58:58
NetinfoBot/1.0/Nutch-0.9 (http://netinfo.bg/netinfobot.html) 2 0% 11.10.2007 11:08:48
testBOT/0.001 2 0% 14.10.2007 23:06:38
woriobot (+http://www.worio.com/) 2 0% 24.10.2007 13:19:01
MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default) 2 0% 06.11.2007 13:11:27
holmes/3.12 (http://morfeo.centrum.cz/bot) 2 0% 10.11.2007 23:55:53
sait/Nutch-0.9 (SAIT Research; http://www.samsung.com) 2 0% 09.11.2007 18:08:40
Scooter/3.3.vscooter 1 0% 10.03.2004 13:34:27
IlTrovatore-Setaccio/1.2 (+http://www.iltrovatore.it/aiuto/faq.html) 1 0% 11.03.2004 11:01:24
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; freenet DSL 1.1; .NET CLR 1.0.3705; MSIECrawler) 1 0% 15.03.2004 14:10:18
Mackster(http://www.click4choice.com ) 1 0% 02.04.2004 09:38:58
augurnfind V-1.8 1 0% 05.04.2004 23:21:26
Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; (R1 1.3)) 1 0% 06.04.2004 23:30:40
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; T312461) 1 0% 08.04.2004 18:55:36
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90; Alexa Toolbar; .NET CLR 1.1.4322) 1 0% 09.04.2004 10:41:59
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; yie6_SBC; YPC 3.0.0; .NET CLR 1.1.4322; MSIECrawler) 1 0% 12.04.2004 22:02:57
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.6) Gecko/20040206 Firefox/0.8 1 0% 13.04.2004 04:21:59
CrawlConvera0.1 ([email protected]) 1 0% 13.04.2004 07:33:59
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; DT; MSIECrawler) 1 0% 15.04.2004 18:58:36
Iltrovatore-Setaccio 1 0% 17.04.2004 12:48:15
htdig/3.1.6 ([email protected]) 1 0% 21.04.2004 16:21:03
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) ([email protected]) 1 0% 27.04.2004 15:59:42
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.4.2.0; MSIECrawler) 1 0% 02.05.2004 12:58:20
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) 1 0% 03.05.2004 16:28:01
ZIPPPCVS/0.24 (ZipppBot/.24; http://www.zippp.net; [email protected]) 1 0% 06.05.2004 23:26:53
semanticdiscovery/0.2(http://www.semanticdiscovery.com/sd/robot.html 1 0% 11.05.2004 17:02:27
lwp-trivial/1.38 1 0% 13.05.2004 16:20:28
deepak-USC/ISI 1 0% 16.05.2004 15:35:22
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; YPC 3.0.0; MSIECrawler) 1 0% 27.05.2004 12:11:47
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Avant Browser [avantbrowser.com]) 1 0% 29.05.2004 16:14:37
Openbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot.php) 1 0% 02.06.2004 22:13:08
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.4.6.0; MSIECrawler) 1 0% 03.06.2004 15:15:06
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; DT; MSIECrawler) 1 0% 04.06.2004 22:10:37
IlTrovatore-Setaccio/0.03-dev (Indexing; http://www.iltrovatore.it/bot.html; [email protected]) 1 0% 09.06.2004 06:47:45
Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.6) Gecko/20040207 Firefox/0.8 1 0% 11.06.2004 20:34:25
Googlebot 1 0% 12.06.2004 15:23:02
Googlebot/2.1 1 0% 12.06.2004 15:32:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.4.6.0; .NET CLR 1.1.4322; MSIECrawler) 1 0% 18.06.2004 14:39:18
Aport 1 0% 22.06.2004 05:10:05
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90; MSIECrawler) 1 0% 25.06.2004 19:29:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts-MyWay; (R1 1.3); MSIECrawler) 1 0% 27.06.2004 15:20:25
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 4.5.0.0; MSIECrawler) 1 0% 29.06.2004 19:06:21
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040206 Firefox/0.8 1 0% 04.07.2004 18:10:21
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7) Gecko/20040626 Firefox/0.9.1 1 0% 09.07.2004 19:20:46
Mackster( http://www.ukwizz.com ) 1 0% 16.07.2004 11:33:46
wwwster/1.2 (Beta) 1 0% 21.07.2004 11:30:50
Mozilla (de) 1 0% 01.08.2004 20:56:09
boitho.com-dc/0.55 ( http://www.boitho.com/dcbot.html ) 1 0% 05.08.2004 06:00:28
larbin_extended ([email protected]) 1 0% 06.08.2004 03:26:05
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; DX-Browser 5.0.0.0; MSIECrawler) 1 0% 10.08.2004 17:40:06
Tutorial Crawler 1.4 1 0% 22.08.2004 09:57:43
Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; freenet DSL 1.00; MSIECrawler) 1 0% 22.08.2004 21:44:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; QXW0339d; .NET CLR 1.1.4322; MSIECrawler) 1 0% 24.08.2004 19:17:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Strato DSL; IBP; .NET CLR 1.1.4322) 1 0% 25.08.2004 05:48:05
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705) 1 0% 01.09.2004 16:43:52
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; DT; MSIECrawler) 1 0% 06.09.2004 09:59:38
CosmixCrawler/0.1 1 0% 16.09.2004 08:47:42
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 1&1 Internet AG by USt; SV1; IBP; .NET CLR 1.1.4322) 1 0% 16.09.2004 16:53:20
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Avant Browser [avantbrowser.com]; SV1) 1 0% 17.09.2004 05:20:48
booch_1.0.7 ([email protected]) 1 0% 22.09.2004 07:24:11
Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1 1 0% 02.10.2004 15:44:35
stat ([email protected]) 1 0% 03.10.2004 12:02:01
TutorGigBot/1.5 ( +http://www.tutorgig.info ) 1 0% 04.10.2004 03:04:53
PhpDig/1.6.2 (PHP; MySql) 1 0% 09.10.2004 06:52:40
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7) Gecko/20040707 Firefox/0.8 1 0% 17.10.2004 22:22:04
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; i-NavFourF; IBP; .NET CLR 1.1.4322) 1 0% 21.10.2004 20:25:02
ASPseek/1.2.11pre 1 0% 25.10.2004 10:02:21
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; IZBSOFT) 1 0% 27.10.2004 08:13:43
http://www.sygol.com 1 0% 04.11.2004 02:46:08
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; IBP; .NET CLR 1.1.4322; Alexa Toolbar) 1 0% 04.11.2004 03:24:26
Ocelli/1.1 (http://www.globalspec.com/Ocelli) 1 0% 05.11.2004 06:37:04
Mozilla/4.0 (compatible; MSIE 5.00; Windows 98) 1 0% 08.11.2004 08:09:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; Alexa Toolbar) 1 0% 10.11.2004 22:27:34
Space2000-Bot (Version: 1.02, powered by www.space2000.de) 1 0% 14.11.2004 17:17:37
EARTHCOM.info/1.6 [www.earthcom.info] 1 0% 21.11.2004 11:40:53
Mozilla/4.7 [en]([email protected]) 1 0% 27.11.2004 07:14:54
Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20040913 Firefox/0.10.1 1 0% 27.11.2004 13:19:09
ES.NET_Crawler/2.0 (http://www.innerprise.net/) 1 0% 28.11.2004 04:09:06
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.3) Gecko/20040910 1 0% 30.11.2004 23:24:43
versus 0.2 (+http://versus.integis.ch) 1 0% 03.12.2004 01:59:51
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0 1 0% 08.12.2004 15:41:46
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (R1 1.3); .NET CLR 1.1.4322; MSIECrawler) 1 0% 10.12.2004 10:04:48
Emmap-client/Alpha 1 0% 20.12.2004 15:45:49
Mozilla/5.0 (compatible; BecomeBot/2.0beta; +http://www.become.com/webmasters.html) 1 0% 27.12.2004 08:16:06
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 1 0% 02.01.2005 00:27:16
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; SV1; .NET CLR 1.1.4322) 1 0% 04.01.2005 14:32:56
http://www.pisoc.com (tarantula) 1 0% 05.01.2005 03:04:27
NutchCVS/0.06-dev (bot; http://www.nebel.de/bot.html; [email protected]) 1 0% 08.01.2005 05:19:43
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; MSIECrawler) 1 0% 11.01.2005 08:52:09
booch_1.0.7 ([email protected]) 1 0% 11.01.2005 09:58:19
Mozilla/5.0 (Windows; U; Win98; de-AT; rv:1.8a5) Gecko/20041122 1 0% 12.01.2005 17:41:40
grub crawler(http://www.grub.org) 1 0% 20.01.2005 00:10:32
GigabotSiteSearch/2.0 (sitesearch.gigablast.com) 1 0% 23.01.2005 01:38:02
ConveraCrawler/0.5 (+http://www 1 0% 23.01.2005 04:59:46
TurnitinBot/2.0 http://www.turnitin.com/robot/crawlerinfo.html 1 0% 23.01.2005 18:35:12
InnerpriseBot/1.0 (http://www.innerprise.com/) 1 0% 26.01.2005 02:20:57
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.5) Gecko/20041122 Firefox/1.0 1 0% 29.01.2005 14:04:00
Opera/7.54u1 (Windows NT 5.1; U) [en] 1 0% 11.02.2005 03:11:27
Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.0rc3) Gecko/20020524 1 0% 19.02.2005 17:59:15
W3C-checklink/4.1 [4.14] libwww-perl/5.803 1 0% 19.02.2005 23:45:12
NetResearchServer/2.9(loopimprovements.com/robot.html) 1 0% 24.02.2005 10:10:33
GoForIt.com 1 0% 25.02.2005 12:31:20
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; QXW03398; MSIECrawler) 1 0% 02.03.2005 12:06:37
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Crazy Browser 1.0.5) 1 0% 03.03.2005 14:01:54
KazoomBot/0.06-dev (Kazoom; http://www.kazoom.ca/bot.html; [email protected]) 1 0% 03.03.2005 18:50:17
Sopheus Project/0.01 (Nutch; http://www.thenetplanet.com; [email protected]) 1 0% 04.03.2005 14:52:43
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 1 0% 04.03.2005 17:56:48
SeznamBot/1.0 1 0% 07.03.2005 23:14:07
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; (R1 1.5); .NET CLR 1.0.3705; .NET CLR 1.1.4322; MSIECrawler) 1 0% 09.03.2005 16:13:19
POE-Component-Client-HTTP/0.65 (perl; N; POE; en; rv:0.650000) 1 0% 14.03.2005 05:41:30
LWP::Simple/5.79 1 0% 15.03.2005 20:29:08
HooWWWer/2.1.0 (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) 1 0% 16.03.2005 14:23:52
Mozilla/6.0 (compatible; arameda.com Spider) 1 0% 17.03.2005 18:09:20
larbin_2.6.3 ([email protected]) 1 0% 18.03.2005 15:23:20
Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) 1 0% 24.03.2005 07:04:19
Mozilla/5.0 (Windows; U; Windows NT 5.0; de-DE; rv:1.7.6) Gecko/20050321 Firefox/1.0.2 1 0% 25.03.2005 17:34:56
YahooSeeker/CafeKelsa-dev (compatible; Konqueror/3.2; FreeBSD ;[email protected] ) (KHTML, like Gecko) 1 0% 30.03.2005 04:21:23
SCrawlTest/CR1 (CWD; http://sproose.com; [email protected]) 1 0% 31.03.2005 02:11:46
abot/0.1 (abot; http://www.abot.com; [email protected]) 1 0% 01.04.2005 06:06:02
AtlocalBot/1.1 +(http://www.atlocal.com/local-web-site-owner.html) 1 0% 02.04.2005 09:43:10
Opera/8.0 (Windows NT 5.1; U; de) 1 0% 03.04.2005 11:13:37
htdig/3.1.6 ([email protected]) 1 0% 03.04.2005 14:52:50
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; tele.ring; .NET CLR 1.1.4322; MSIECrawler) 1 0% 07.04.2005 09:24:16
http://www.almaden.ibm.com/cs/crawler [bc35] 1 0% 07.04.2005 16:48:06
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050404 Firefox/1.0.3 1 0% 09.04.2005 15:46:41
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.6) Gecko/20050226 Firefox/1.0.1 1 0% 09.04.2005 18:05:01
NetResearchServer/3.4(loopimprovements.com/robot.html) 1 0% 09.04.2005 18:48:56
MultiText/0.1 1 0% 12.04.2005 01:33:50
webcrawl.net 1 0% 12.04.2005 09:32:39
Knowledge.com/0.2 1 0% 14.04.2005 14:58:13
bscrawler_0.1_alfa ([email protected]) 1 0% 18.04.2005 16:10:50
( Robots.txt Validator http://www.searchengineworld.com/cgi-bin/robotcheck.cgi ) 1 0% 22.04.2005 06:29:57
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7) Gecko/20040803 Firefox/0.9.3 1 0% 24.04.2005 13:26:57
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de-DE; rv:1.7.5) Gecko/20041108 Firefox/1.0 1 0% 29.04.2005 18:18:13
IlTrovatore-Setaccio/1.2 (It-bot;compatible;MSIE 6.0;Mozilla/4.0; http://www.iltrovatore.it/bot.html; [email protected]) 1 0% 30.04.2005 00:47:44
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3 1 0% 02.05.2005 02:55:17
Java/1.4.2_03 1 0% 02.05.2005 15:24:58
Mozilla/5.0 (compatible; BecomeBot/1.85; MSIE 6.0 compatible; +http://www.become.com/site_owners.html) 1 0% 03.05.2005 11:44:27
ConveraCrawler/0.7 (+http://www.authoritativeweb.com/crawl) 1 0% 10.05.2005 15:36:35
findimbot/1.8.3 (http://www.findim.net/findimbot.php) 1 0% 13.05.2005 15:08:42
larbin ([email protected]) 1 0% 14.05.2005 12:31:30
larbin_2.6.3 ([email protected]) 1 0% 17.05.2005 14:28:03
Mozilla/5.0 (compatible; heritrix/1.4.0 +http://www.basistech.com/crawlinfo.html) 1 0% 27.05.2005 02:02:46
Wget/1.9 1 0% 27.05.2005 09:31:17
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; DT; SV1; MSIECrawler) 1 0% 29.05.2005 20:23:26
ConveraCrawler/0.8 (+http://www.authoritativeweb.com/crawl) 1 0% 31.05.2005 01:48:52
mozilla/5.0 (compatible; genevabot +http://www.healthdash.com) 1 0% 02.06.2005 18:36:38
Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/412 (KHTML, like Gecko) Safari/412 1 0% 03.06.2005 16:18:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.1.4322; MSIECrawler) 1 0% 08.06.2005 20:14:01
simpsons-fc.de (compatible; MSIE 6.0; Windows NT 5.1; SV1; i-NavFourF; .NET CLR 1.0.3705; .NET CLR 1.1.4322) 1 0% 09.06.2005 21:07:21
larbin_2.6.3 ([email protected]) 1 0% 13.06.2005 09:30:48
ObjectsSearch/0.06 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; [email protected]) 1 0% 16.06.2005 20:55:48
savvybot/0.2 1 0% 23.06.2005 10:32:09
Mozilla/5.0 (Macintosh; U; PPC Mac OS X; de-de) AppleWebKit/312.1 (KHTML, like Gecko) Safari/312 1 0% 26.06.2005 13:51:26
axadine/ (Axadine Crawler; http://www.axada.de/; ) 1 0% 28.06.2005 18:49:18
Minuteman/0.3 1 0% 01.07.2005 00:41:44
Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot .php) 1 0% 02.07.2005 02:38:57
NutchTestBot/0.0 (Nutch Test; ; [email protected]) 1 0% 02.07.2005 05:46:27
silk/2.4 1 0% 06.07.2005 01:13:14
Mozilla/5.0 (compatible; heritrix/1.5.0 +http://www.offis.de/) 1 0% 08.07.2005 05:30:51
Mozilla/5.0 (compatible; heritrix/1.5.0 +http://www.offis.de) 1 0% 09.07.2005 03:37:10
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-DE; rv:1.7.8) Gecko/20050511 Firefox/1.0.4 1 0% 10.07.2005 14:10:11
OmniExplorer_Bot/2.5a (+http://www.omni-explorer.com) WorldIndexer 1 0% 11.07.2005 09:03:21
Akwan Crawler 0.1 1 0% 12.07.2005 06:55:44
Ultraknowledge Spider 1 0% 12.07.2005 09:46:07
OmniExplorer_Bot/2.9a (+http://www.omni-explorer.com) WorldIndexer 1 0% 12.07.2005 09:47:29
OmniExplorer_Bot/2.42b (+http://www.omni-explorer.com) WorldIndexer 1 0% 13.07.2005 20:24:49
OmniExplorer_Bot/2.44c (+http://www.omni-explorer.com) WorldIndexer 1 0% 13.07.2005 23:36:18
OmniExplorer_Bot/2.47 (+http://www.omni-explorer.com) WorldIndexer 1 0% 14.07.2005 22:44:34
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; IBP) 1 0% 15.07.2005 03:54:30
OmniExplorer_Bot/2.65 (+http://www.omni-explorer.com) WorldIndexer 1 0% 16.07.2005 00:52:35
OmniExplorer_Bot/2.67 (+http://www.omni-explorer.com) WorldIndexer 1 0% 16.07.2005 16:58:29
Orbiter/T-2.0 (+http://www.dailyorbit.com/bot.htm) 1 0% 16.07.2005 21:21:45
OmniExplorer_Bot/2.69 (+http://www.omni-explorer.com) WorldIndexer 1 0% 17.07.2005 04:12:47
OmniExplorer_Bot/2.71 (+http://www.omni-explorer.com) WorldIndexer 1 0% 18.07.2005 06:23:42
Filangy/1.01 (Filangy; http://www.filangy.com/filangyinfo.jsp?inc=robots.jsp; [email protected]) 1 0% 18.07.2005 22:01:16
OmniExplorer_Bot/2.73 (+http://www.omni-explorer.com) WorldIndexer 1 0% 19.07.2005 00:26:32
Mozilla/4.0 (compatible) 1 0% 19.07.2005 18:49:24
findlinks/0.944 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 20.07.2005 17:51:50
OmniExplorer_Bot/2.86b (+http://www.omni-explorer.com) WorldIndexer 1 0% 22.07.2005 23:16:09
OmniExplorer_Bot/2.92 (+http://www.omni-explorer.com) WorldIndexer 1 0% 24.07.2005 00:29:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;) 1 0% 24.07.2005 10:22:06
Mozilla/5.0 (X11; U; Linux i686;en-US; rv:1.2.1) Gecko/20030225 1 0% 27.07.2005 21:49:44
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Tablet PC 1.7; MSIECrawler) 1 0% 30.07.2005 16:41:31
Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html) 1 0% 01.08.2005 20:48:30
findlinks/0.945 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 02.08.2005 19:27:44
boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html ) 1 0% 05.08.2005 22:23:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Jacksonville Job Corps Center; MSIECrawler) 1 0% 07.08.2005 00:40:47
FAST Enterprise Crawler 6 used by Fast Search ([email protected]) 1 0% 09.08.2005 11:54:23
searchbot ([email protected]) 1 0% 10.08.2005 23:52:51
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; QXW03416; MSIECrawler) 1 0% 13.08.2005 11:55:06
DataparkSearch/4.32 (+http://www.dataparksearch.org/) 1 0% 15.08.2005 11:25:33
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; H010818; S.N.O.W.2; .NET CLR 1.1.4322; MSIECrawler) 1 0% 16.08.2005 08:45:44
8 1 0% 16.08.2005 19:32:18
G~9 1 0% 16.08.2005 21:10:03
P:zx79z 1 0% 17.08.2005 18:41:09
/necacheH 1 0% 17.08.2005 21:05:42
T_0턈 1 0% 18.08.2005 21:44:06
0xGl 1 0% 18.08.2005 23:50:25
Helix/1.2 (+http://www.sitesearch.ca/helix/) 1 0% 19.08.2005 02:16:10
ΐ}r 1 0% 19.08.2005 05:04:33
larbin_2.6.3 ([email protected]) 1 0% 20.08.2005 20:14:20
InfociousBot (+http://corp.infocious.com/tech_crawler.php) 1 0% 22.08.2005 11:39:38
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; QXW0339m; MSIECrawler) 1 0% 22.08.2005 18:57:29
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Wanadoo042NLPP01HCFFFFFFEC; SV1; .NET CLR 1.1.4322; MSIECrawler) 1 0% 28.08.2005 21:56:15
schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; Contact: [email protected];) 1 0% 01.09.2005 11:52:23
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Arcor 5.004; .NET CLR 1.1.4322; MSIECrawler) 1 0% 07.09.2005 16:12:15
NuSearch Spider www.nusearch.com 1 0% 17.09.2005 07:50:45
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; MSIECrawler) 1 0% 20.09.2005 15:33:47
Mozilla/5.0 (compatible; heritrix/1.0 +http://metacarta.com) 1 0% 21.09.2005 07:29:11
findlinks/0.975 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 22.09.2005 15:24:01
HeinrichderMiragoRobot (http://www.miragorobot.com/scripts/deinfo.asp) 1 0% 25.09.2005 08:09:09
WebSearch.COM.AU/4.0 (The Australian Search Engine; http://WebSearch.COM.AU; [email protected]) 1 0% 27.09.2005 06:21:58
LocalcomBot/1.2 (+http://www.local.com/bot.htm) 1 0% 27.09.2005 22:03:48
********** (***************; ********; Geht keinen was an!; .NET CLR 1.1.4322; MSIECrawler) 1 0% 29.09.2005 13:09:17
ICrawl ([email protected]) 1 0% 29.09.2005 18:24:49
EverbeeCrawler 1 0% 30.09.2005 13:47:20
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Strato DSL; .NET CLR 1.1.4322; Alexa Toolbar) 1 0% 03.10.2005 23:56:46
Wget/1.9+cvs-dev 1 0% 04.10.2005 11:30:12
larbin_2.6.3 ([email protected]) 1 0% 07.10.2005 10:49:07
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; i-NavFourF; MSIECrawler) 1 0% 07.10.2005 11:34:08
Mozilla/5.0 (compatible; heritrix/1.4.0 +http://www.offis.de) 1 0% 07.10.2005 21:39:31
NutchCVS/0.06-dev (http://www.nutch.org/docs/en/bot.html; [email protected]) 1 0% 22.10.2005 06:00:29
htdig/3.1.6 ([email protected]) 1 0% 25.10.2005 08:21:12
FAST Enterprise Crawler 6 used by Crawler GonzoP1 DE ([email protected]) 1 0% 26.10.2005 04:23:23
WebMiner (Web Miner; http://64.124.122.252/feedback.html) 1 0% 27.10.2005 18:06:44
... 1 0% 30.10.2005 02:23:05
FyberSpider (+http://www.fybersearch.com/fyberspider.php) 1 0% 31.10.2005 06:37:36
DNSGroup/0.1 (DNS Group Crawler; http://www.dnsgroup.com/; [email protected]) 1 0% 01.11.2005 12:45:12
Ipselonbot/0.46-beta (Ipselon; http://www.ipselon.com/intl/en/ipselonbot.html; [email protected]) 1 0% 01.11.2005 13:42:13
Orbiter (+http://www.dailyorbit.com/bot.htm) 1 0% 01.11.2005 19:50:22
Jakarta Commons-HttpClient/3.0-rc3 1 0% 02.11.2005 09:36:40
simon/2.2 1 0% 08.11.2005 02:39:02
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_05; Europe/de) 1 0% 08.11.2005 17:41:26
OmniExplorer_Bot/4.32 (+http://www.omni-explorer.com) WorldIndexer 1 0% 11.11.2005 00:17:09
findlinks/1.06 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 12.11.2005 02:22:54
VSE/1.0 ([email protected]) 1 0% 13.11.2005 04:43:12
Mozilla/4.0 (compatible; MSIE 6.0 Windows NT5.1) ([email protected]) 1 0% 13.11.2005 11:39:32
Ipselonbot/0.53-beta (Ipselon; http://www.ipselon.com/intl/en/ipselonbot.html; [email protected]) 1 0% 15.11.2005 06:12:35
VSE/1.0 ([email protected]) 1 0% 16.11.2005 01:45:36
yacy (www.yacy.net; v20040602; ppc Mac OS X 10.4.3; java 1.4.2_09; Europe/de) 1 0% 17.11.2005 15:38:10
findlinks/1.0.8 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 19.11.2005 18:08:00
Mozilla/5.0 (compatible; heritrix/1.5 +http://www.metacarta.com) 1 0% 20.11.2005 00:23:55
Ipselonbot/1.0-beta (+; http://www.ipselon.com/intl/en/ipselonbot.html) 1 0% 28.11.2005 04:00:47
FAST Enterprise Crawler 6 used by HBZ ([email protected]) 1 0% 02.12.2005 20:51:06
larbin_2.6.3 ([email protected]) 1 0% 04.12.2005 20:37:46
yacy (www.yacy.net; v20040602; i386 Linux 2.6.12; java 1.5.0_06; Europe/de) 1 0% 14.12.2005 15:37:20
Mozilla/5.0 (compatible; Theophrastus/1.2; +http://users.cs.cf.ac.uk/N.A.Smith/theophrastus.php) 1 0% 16.12.2005 16:37:40
ccubee/3.7 1 0% 16.12.2005 18:18:51
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Strato DSL; .NET CLR 1.1.4322; MSIECrawler) 1 0% 18.12.2005 13:07:59
g2Crawler ([email protected]) 1 0% 20.12.2005 02:13:57
VORTEX/1.0 (+http://marty.anstey.ca/robots/vortex/) 1 0% 20.12.2005 14:57:49
larbin_2.6.3 ([email protected]) 1 0% 22.12.2005 16:42:34
silk/1.0 (+http://www.slider.com/silk.htm)/3.7 1 0% 27.12.2005 04:05:03
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; SV1; MSIECrawler) 1 0% 27.12.2005 10:36:56
yacy (www.yacy.net; v20040602; i386 Linux 2.6.14-skas3-v8.2; java 1.5.0_06; Europe/en) 1 0% 29.12.2005 12:31:13
EARTHCOM.info/1.99 [www.earthcom.info] 1 0% 04.01.2006 00:18:24
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_04; Europe/de) 1 0% 04.01.2006 17:43:46
RedCell/0.1 (RedCell; telegenetic.net/bot.html; lhall_at_telegenetic.net) 1 0% 04.01.2006 22:47:08
gsa-crawler (Enterprise; MID-02068; [email protected],[email protected]) 1 0% 05.01.2006 00:14:03
CSCrawler | homepage: http://www.kde.cs.uni-kassel.de/lehre/ss2005/googlespam/crawler.html RPT-HTTPClient/0.3-3 1 0% 06.01.2006 04:46:15
WIRE/0.10 (Linux; i686; Bot,Robot,Spider,Crawler) 1 0% 06.01.2006 14:54:47
plinki/0.1 (you got plinked! (thats a good thing..); http://www.plinki.com; [email protected]) 1 0% 09.01.2006 05:07:21
xirq/0.1-dev (xirq; http://www.xirq.com; [email protected]) 1 0% 12.01.2006 11:16:41
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Arcor 5.003; MSIECrawler) 1 0% 13.01.2006 01:01:57
findlinks/1.1-a9 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 13.01.2006 13:49:42
WIRE/0.10 (Linux; i686; bot42) 1 0% 14.01.2006 11:13:38
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8) Gecko/20051111 Firefox/1.5 1 0% 25.01.2006 13:09:05
Mozilla/5.0 (Windows;) NimbleCrawler 1.12 obeys UserAgent NimbleCrawler For problems contact: [email protected] 1 0% 27.01.2006 14:34:14
findlinks/1.1.1-a1 (+http://wortschatz.uni-leipzig.de/findlinks/) 1 0% 29.01.2006 23:21:31
yacy (www.yacy.net; v20040602; i386 Linux 2.6.13-15.7-default; java 1.4.2_06; Europe/en) 1 0% 02.02.2006 08:49:41
ObjectsSearch/0.2 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; [email protected]) 1 0% 05.02.2006 00:24:44
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MSIECrawler) 1 0% 05.02.2006 01:21:11
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 1 0% 09.02.2006 23:16:13
larbin_test ([email protected]) 1 0% 10.02.2006 18:47:13
yacy (www.yacy.net; v20040602; i386 Linux 2.4.20-021stab028.17.777-enterprise; java 1.5.0_06; Europe/en) 1 0% 10.02.2006 18:49:43
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; HbTools 4.7.2; .NET CLR 2.0.50727; MSIECrawler) 1 0% 11.02.2006 19:37:40
Mozilla/5.0 (Windows;) NimbleCrawler 1.13 obeys UserAgent NimbleCrawler For problems contact: [email protected] 1 0% 17.02.2006 06:13:23
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; {F1554AEC-66D3-AFA7-3574-608C988FDDC9}; snprtz|dialno; .NET CLR 1.1.4322; MSIECrawler) 1 0% 17.02.2006 22:49:33
suchpad/1.0 (+http://www.suchpad.de) 1 0% 19.02.2006 17:49:39
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 1 0% 19.02.2006 20:02:40
HooWWWer/2.1.3 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) 1 0% 23.02.2006 02:16:51
yacy (www.yacy.net; v20040602; x86 Windows 2000 5.0; java 1.5.0_06; Atlantic/de) 1 0% 25.02.2006 11:40:10
Mozilla/5.0 (Windows;) NimbleCrawler 1.14 obeys UserAgent NimbleCrawler For problems contact: [email protected] 1 0% 28.02.2006 01:47:14
BurstFind Crawler 1.0/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 1 0% 01.03.2006 06:56:56
FAST Enterprise Crawler 6 used by Schibsted Sok ([email protected]) 1 0% 02.03.2006 10:18:21
Mozilla/5.0 (Windows; U; Win 9x 4.90; ru; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 1 0% 06.03.2006 22:22:24
yacy (www.yacy.net; v20040602; i386 Linux 2.4.27-2-386; java 1.5.0_04; Europe/en) 1 0% 10.03.2006 14:07:55
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8) Gecko/20051107 Firefox/1.5 1 0% 11.03.2006 15:16:41
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 1 0% 14.03.2006 15:38:30
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.1) Gecko/20060111 Spacefinch Firefox/1.5.0.1 1 0% 16.03.2006 09:57:53
yacy (www.yacy.net; v20040602; i386 Linux 2.6.10; java 1.5.0_06; US/en) 1 0% 18.03.2006 08:32:19
Firefox_1.0.6 ([email protected]) 1 0% 19.03.2006 18:35:30
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 1 0% 20.03.2006 10:34:30
hl_ftien_spider 1 0% 26.03.2006 23:31:08
Blaiz-Bee/2.00.953 (+http://www.blaiz.net) 1 0% 27.03.2006 03:33:53
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Webblaireau (null - #verison#) Firefox/1.5.0.1 1 0% 01.04.2006 17:45:41
updated/0.1-beta (updated; http://www.updated.com; [email protected]) 1 0% 06.04.2006 18:20:55
NextGenSearchBot 1 (for information visit http://www.zoominfo.com/NextGenSearchBot) 1 0% 07.04.2006 20:08:49
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 2.0.50727; MSIECrawler) 1 0% 11.04.2006 07:08:58
noyona_0_1 1 0% 13.04.2006 00:57:06
Mozdex/0.7.2 (Mozdex; http://www.mozdex.com/bot.html; [email protected]) 1 0% 13.04.2006 02:09:29
Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 1 0% 17.04.2006 01:33:23
RAMPyBot - www.giveRAMP.com/1.0 (RAMPyBot - www.giveRAMP.com; http://www.giveramp.com/bot.html; [email protected]) 1 0% 19.04.2006 19:01:22
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 1 0% 20.04.2006 00:25:18
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 1 0% 21.04.2006 13:40:18
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 1 0% 21.04.2006 14:54:48
kyluka-get 1 0% 22.04.2006 02:41:19
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.2) Gecko/20060308 Firefox/1.5.0.2 1 0% 22.04.2006 14:19:36
NPBot/3 (NPBot; http://www.nameprotect.com; [email protected]) 1 0% 24.04.2006 06:28:15
Mozilla/5.0 (compatible; heritrix/1.6.0 +http://www.creativeandlogics.com/) 1 0% 25.04.2006 06:22:28
RAMPyBot - www.giveRAMP.com/0.1 (RAMPyBot - www.giveRAMP.com; http://www.giveramp.com/bot.html; [email protected]) 1 0% 26.04.2006 02:19:48
WIRE/0.11 (Linux; i686; Bot42) 1 0% 27.04.2006 07:03:48
gsa-crawler (Enterprise; GIX-02057; [email protected]) 1 0% 05.05.2006 04:13:17
Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 1 0% 05.05.2006 10:43:24
GeoBot/version 1.0 1 0% 06.05.2006 21:28:24
Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 1 0% 08.05.2006 19:26:11
Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 1 0% 09.05.2006 10:48:45
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr-FR; rv:1.7.12) Gecko/20050919 Firefox/1.0.7 1 0% 11.05.2006 08:05:46
Crawllybot/0.1 (Crawllybot; +http://www.crawlly.com; [email protected]) 1 0% 13.05.2006 03:27:17
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; {6529BE05-8634-35BA-65DA-B762FB78FE4A}; .NET CLR 1.1.4322; MSIECrawler) 1 0% 16.05.2006 20:08:12
Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://www.krugle.com/crawler/info.html; [email protected]) 1 0% 17.05.2006 19:22:52
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 1.0.3705; MSIECrawler) 1 0% 18.05.2006 22:50:49
Skywalker/0.1 (Skywalker; anonymous; anonymous) 1 0% 20.05.2006 15:53:44
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 1.1.4322; MSIECrawler) 1 0% 20.05.2006 19:04:51
Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.3) Gecko/20060426 Firefox/1.5.0.3 1 0% 24.05.2006 11:01:44
WinkBot/0.06 (Wink.com search engine web crawler; http://www.wink.com/Wink:WinkBot; [email protected]) 1 0% 24.05.2006 14:28:03
Mozilla/4.0 (compatible; MSIE 9.42; Windows 3.11) 1 0% 25.05.2006 15:38:18
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Avant Browser; Avant Browser; .NET CLR 2.0.50727; MSIECrawler) 1 0% 25.05.2006 21:28:35
larbin_2.6.3 ([email protected]) 1 0% 25.05.2006 22:41:28
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Q312461; .NET CLR 1.0.3705; .NET CLR 1.1.4322) 1 0% 28.05.2006 18:26:52
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.3) Gecko/20060523 Ubuntu/dapper Firefox/1.5.0.3 1 0% 01.06.2006 22:13:45
wwwster/1.4 (Beta, mailto:[email protected]) 1 0% 02.06.2006 07:57:10
gzip(gfe) (via translate.google.com) 1 0% 02.06.2006 11:18:23
Mozilla/5.0 (X11; U; Linux i686; rv:1.8.0.3) Gecko/20060503 Firefox/1.5.0.3 (Swiftfox) 1 0% 04.06.2006 22:31:37
DataparkSearch/4.40.1 (+http://www.dataparksearch.org/) 1 0% 07.06.2006 06:02:56
NG-Search/0.90 (NG-SearchBot; http://www.ng-search.com/; ) 1 0% 12.06.2006 11:57:19
Anonymous/0.0 (Anonymous; http://www.anonymous.com; [email protected]) 1 0% 13.06.2006 01:28:28
Yahoo-Blogs/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; http://help.yahoo.com/help/us/ysearch/crawling/crawling-02.html ) 1 0% 16.06.2006 12:09:29
suchpadbot/1.0 (+http://www.suchpad.de/bot/) 1 0% 17.06.2006 23:52:57
Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 1 0% 21.06.2006 19:51:22
Mozilla/5.0 (Windows; U; Win 9x 4.90; de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 1 0% 22.06.2006 22:06:23
Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 1 0% 26.06.2006 10:15:04
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 1 0% 28.06.2006 16:30:20
Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.myfamilyinc.com) 1 0% 29.06.2006 18:32:28
yacy (www.yacy.net; v20040602; i386 Linux 2.6.8-2-686; java 1.5.0_07; GMT+01:00/de) 1 0% 30.06.2006 17:44:39
yacy (www.yacy.net; v20040602; i386 Linux 2.6.8-022stab070.5-enterprise; java 1.4.2-03; Europe/en) 1 0% 30.06.2006 21:34:32
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; ezn-ie5-r77-32; FDM; MSIECrawler) 1 0% 02.07.2006 21:30:12
Exabot-Test/1.0 1 0% 03.07.2006 12:26:52
Mozilla/5.0 (compatible; heritrix/1.6.0 +http://www.worio.com/) 1 0% 05.07.2006 23:00:27
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 2.0.50727; MSIECrawler) 1 0% 08.07.2006 00:14:48
www.adressendeutschland.de 1 0% 08.07.2006 14:03:46
Nusearch Spider (www.nusearch.com) 1 0% 09.07.2006 00:59:03
Mozilla/4.0 (compatible; MSIE 6.0 Windows NT5.1) ([email protected]) 1 0% 10.07.2006 05:51:40
Zeus 15187 Webster Pro V2.9 Win32 1 0% 10.07.2006 11:45:44
FeedChecker/0.01 1 0% 11.07.2006 20:24:02
gonzo1[T] mailto:[email protected] 1 0% 12.07.2006 10:50:07
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.4) Gecko/20060608 Ubuntu/dapper-security Firefox/1.5.0.4 1 0% 12.07.2006 12:07:16
Java/1.4.2_09 1 0% 13.07.2006 17:32:59
LocalcomBot/1.2.2 (+http://www.local.com/bot.htm) 1 0% 16.07.2006 02:15:43
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; HbTools 4.7.1; MSIECrawler) 1 0% 17.07.2006 18:15:27
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; SIMBAR Enabled; .NET CLR 1.1.4322; .NET CLR 2.0.50727; MSIECrawler) 1 0% 20.07.2006 18:48:10
traazibot/testengine (+http://www.traazi.de) 1 0% 21.07.2006 23:44:42
Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4 1 0% 28.07.2006 11:11:45
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; de; rv:1.8.0.5) Gecko/20060723 Firefox/1.5.0.5 1 0% 01.08.2006 21:09:23
KRetrieve/1.1/dbsearchexpert.com 1 0% 04.08.2006 17:03:40
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; AtHome033; .NET CLR 1.1.4322; MSIECrawler) 1 0% 06.08.2006 11:35:35
Mozilla/5.0 (X11; ; Linux i686; rv:1.8.0.5) Gecko/20060731 1 0% 08.08.2006 16:46:14
Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.0.6) Gecko/20060728 SUSE/1.5.0.6-0.1 Firefox/1.5.0.6 1 0% 09.08.2006 05:27:25
Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 1 0% 09.08.2006 15:39:19
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.6) Gecko/20060807 Firefox/1.5.0.6 1 0% 09.08.2006 21:58:39
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_07; Europe/de) 1 0% 10.08.2006 01:09:02
Mozilla/5.0 (compatible; googlebot/2.1; +http://www.google.com/bot.html) ([email protected]) 1 0% 17.08.2006 11:20:20
BilgiBetaBot/0.8-dev (bilgi.com (Beta) ; http://lucene.apache.org/nutch/bot.html; [email protected]) 1 0% 17.08.2006 23:56:57
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.5) Gecko/20060719 Firefox/1.5.0.5 1 0% 18.08.2006 01:36:48
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Smart Explorer v6.0 ( Evaluation Period ); HbTools 4.8.0; MSIECrawler) 1 0% 20.08.2006 13:19:52
MyEngines-Bot (Version: 1.04, powered by www.d-o-m-a-i-n.de +http://www.d-o-m-a-i-n.de/pfadzurbotseite/bot.html) 1 0% 20.08.2006 19:34:51
Mozilla/5.0 (Windows; U; Windows NT 5.1; nl-NL; rv:1.7.5) Gecko/20041202 Firefox/1.0 1 0% 23.08.2006 21:27:29
ItsyBitsySpidr/0.0.8c (http://blogmonitor.de/misc/itsybitsyspidr) 1 0% 24.08.2006 16:05:07
MedusaCVS/0.1.0 (Medusa; http://www.site.com/medusa/bot.html; [email protected]) 1 0% 25.08.2006 13:20:09
yacy (www.yacy.net; v20040602; i386 Linux 2.6.9-022stab078.14-enterprise; java 1.5.0_07; Europe/en) 1 0% 29.08.2006 18:01:06
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; de; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 1 0% 01.09.2006 15:33:38
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8) Gecko/20051128 SUSE/1.5-0.1 Firefox/1.5 1 0% 01.09.2006 21:39:27
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.5) Gecko/20060728 Firefox/1.5.0.5 1 0% 03.09.2006 14:12:34
TMCrawler 1 0% 04.09.2006 20:37:52
sgckbrsu wnRfnbfskejlrhRd 1 0% 05.09.2006 07:52:26
vvhjbghc9fnl9yyxcrdsf k lyvpfRgcm9m 1 0% 05.09.2006 08:06:41
ItsyBitsy/0.0.8c (http://blogmonitor.de/misc/itsybitsy) 1 0% 07.09.2006 15:39:54
spglogxtprip1g 1ude mkxj 1 0% 08.09.2006 09:58:48
Wget/1.10 1 0% 10.09.2006 15:02:22
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.6) Gecko/20060728 (CK-obertuerke) Firefox/1.5.0.6 1 0% 12.09.2006 20:18:27
BilgiBetaBot/0.8-dev/0.8-dev (bilgi.com (Beta); http://lucene.apache.org/nutch/bot.html; [email protected]) 1 0% 15.09.2006 22:20:23
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5 #PHI#~ 1 0% 22.09.2006 12:27:15
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; iebar; InfoPath.1; MSIECrawler) 1 0% 22.09.2006 22:33:11
Mozilla/4.0 (compatible; Vagabondo/4.0Beta; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/) 1 0% 24.09.2006 17:52:58
SrevBot 1 0% 27.09.2006 06:40:30
miniRank/2.0 (miniRank; http://minirank.com/; website ranking engine) 1 0% 29.09.2006 07:23:12
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; onlineTV; www.cdesign.de; .NET CLR 2.0.50727; InfoPath.1; MSIECrawler) 1 0% 01.10.2006 13:19:22
HooWWWer/2.3.0 (debugging run) (+http://cosco.hiit.fi/search/hoowwwer/ | mailto:crawler-infohiit.fi) 1 0% 02.10.2006 20:18:03
msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm) 1 0% 03.10.2006 08:08:42
yacy (www.yacy.net; v20040602; ppc Linux 2.6.15-27-powerpc; java 1.5.0; Europe/de) 1 0% 04.10.2006 06:29:52
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.7) Gecko/20060921 Ubuntu/dapper-security Firefox/1.5.0.7 1 0% 04.10.2006 15:16:48
Shim-Crawler(Mozilla-compatible; http://www.logos.ic.i.u-tokyo.ac.jp/crawl/; [email protected]) 1 0% 04.10.2006 17:03:30
yacy (www.yacy.net; v20040602; i386 Linux 2.6.16.13-xenU; java 1.5.0_06; UTC/en) 1 0% 04.10.2006 20:31:14
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 2.0.50727; MSIECrawler) 1 0% 05.10.2006 10:01:41
Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 05.10.2006 19:28:57
Mozilla/5.0 (compatible; nextthing.org/1.0; +http://www.nextthing.org/bot) 1 0% 07.10.2006 05:07:30
Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 09.10.2006 14:34:58
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 12.10.2006 22:24:29
Mozilla/5.0 (Macintosh; U; Intel Mac OS X; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 15.10.2006 14:44:32
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2) 1 0% 17.10.2006 00:19:59
yacy (www.yacy.net; v20040602; ppc Mac OS X 10.4.8; java 1.5.0_06; Europe/de) 1 0% 17.10.2006 00:29:20
yacy (www.yacy.net; v20040602; i386 Mac OS X 10.4.8; java 1.5.0_06; Europe/de) 1 0% 17.10.2006 16:41:47
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 17.10.2006 18:58:39
Mozilla/5.0 (Windows; U; Windows NT 5.1; nl; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6 1 0% 18.10.2006 19:36:26
Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 18.10.2006 20:08:57
Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.0.7) Gecko/20060911 SUSE/1.5.0.7-1.2 Firefox/1.5.0.7 1 0% 19.10.2006 14:14:34
nicebot 1 0% 30.10.2006 21:41:40
Combine/3 http://combine.it.lth.se/ 1 0% 02.11.2006 12:45:12
Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 1 0% 03.11.2006 15:11:50
MJ12bot/v1.1.0 (http://majestic12.co.uk/bot.php?+)&I=6FEB9BB768CD40E33BD285923194C7EA -543D4D99456BF56A-F0E09BC7C7A3205D 1 0% 05.11.2006 07:30:03
Szukacz/1.5 (robot; www.szukacz.pl/jakdzialarobot.html; [email protected]) 1 0% 07.11.2006 00:22:14
Nutch/Nutch-0.8.1 1 0% 07.11.2006 02:43:27
Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1) Gecko/20061010 Firefox/2.0 1 0% 09.11.2006 00:08:31
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 1 0% 11.11.2006 02:54:49
&I=901F26CF39D60DF16A1C0D3498941C9B-28FDC445CFE66EA8-991F73320F8757D6- www.christoph-bachner.net 1 0% 21.12.2006 12:56:56
Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.ancestry.com/learn/bot.aspx; [email protected]) 1 0% 22.12.2006 21:24:09
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1 1 0% 23.12.2006 01:20:13
yacy (www.yacy.net; v20040602; i386 Linux 2.6.18-3-686; java 1.5.0_08; GMT+01:00/de) 1 0% 26.12.2006 11:26:39
Vacobot; (+http://vaco.ws/bot.html) 1 0% 26.12.2006 14:00:14
yacy (www.yacy.net; v20040602; i386 Linux 2.6.17-10-386; java 1.5.0_08; Europe/de) 1 0% 28.12.2006 12:01:53
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; freenet.de frndial 5.0; MSIECrawler) 1 0% 30.12.2006 16:46:08
Mozilla/5.0 (compatible; Heritrix/1.8.0 +http://www.hanzoarchives.com) 1 0% 31.12.2006 12:06:47
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; SIMBAR Enabled; SIMBAR=0; SIMBAR={69E4F465-9BFF-4b8b-A3B3-D99203D8AFE9}; HbTools 4.7.7; .NET CLR 1.1.4322; MEGAUPLOAD 1.0; MSIECrawler) 1 0% 01.01.2007 20:41:01
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_09; Europe/de) 1 0% 04.01.2007 21:05:58
BlogMyWay.Net/BlogMyWay-0.8.1 ([email protected]) 1 0% 12.01.2007 11:40:29
yacy (www.yacy.net; v20040602; i386 Linux 2.6.16-rc6-060319a; java 1.4.2-03; Europe/en) 1 0% 13.01.2007 19:37:00
kitty/Nutch-0.8.1 (kitty; http://lucene.apache.org/nutch/; [email protected]) 1 0% 16.01.2007 19:37:35
&I=3BA4B6B151D9F2063F2C256D8111A7D8-28FDC445CFE66EA8-991F73320F8757D6- www.christoph-bachner.net 1 0% 18.01.2007 14:49:54
iSearch/2.16 1 0% 21.01.2007 19:15:09
yacy (www.yacy.net; v20040602; i386 Linux 2.6.17-10-generic; java 1.4.2-03; Europe/en) 1 0% 21.01.2007 21:45:44
kobe_university_search/Nutch-0.8.1 (kobe_university_search; http://www.ai.cs.scitec.kobe-u.ac.jp/~yuu/file/kobe_university.txt; yuu at ai dot cs dot scitec dot kobe-u dot ac dot jp) 1 0% 24.01.2007 19:00:25
Mozilla/5.0 (compatible; seek365 http://www.seek365.de) 1 0% 28.01.2007 08:28:13
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; InfoPath.1; MSIECrawler) 1 0% 29.01.2007 16:51:15
gonzo1[D] +http://www.suchen.de/popups/faq.jsp 1 0% 30.01.2007 00:34:17
Mozilla/5.0 (compatible; MojeekBot/2.0; http://www.mojeek.com/bot.html) 1 0% 01.02.2007 06:51:46
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; MSIECrawler) 1 0% 07.02.2007 18:45:33
Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; FunWebProducts; MSIECrawler) 1 0% 11.02.2007 10:51:22
hl_ftien_spider_v1.1 1 0% 11.02.2007 17:26:24
FAST Enterprise Crawler 6 used by Pages Jaunes ([email protected]) 1 0% 12.02.2007 12:20:51
Mozdex/0.7.1 (Mozdex; http://www.mozdex.com/bot.html; [email protected]) 1 0% 12.02.2007 12:35:58
yacy (www.yacy.net; v20040602; i386 FreeBSD 5.5-RC1; java 1.4.2-p7; Europe/en) 1 0% 14.02.2007 15:47:23
&I=B179EAEAA296AAEB8E0E041B7F27698D-28FDC445CFE66EA8-991F73320F8757D6- www.christoph-bachner.net 1 0% 16.02.2007 14:51:11
yacy (www.yacy.net; v20040602; i386 Linux 2.4.27-2-386; java 1.5.0_11; Europe/de) 1 0% 17.02.2007 15:48:28
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; FDM; MSIECrawler) 1 0% 18.02.2007 15:00:03
KBeeBot/0.5 Robot 1 0% 18.02.2007 17:23:28
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727; MSIECrawler) 1 0% 19.02.2007 12:43:13
yacy (www.yacy.net; v20040602; i386 Linux 2.6.15.4; java 1.5.0_06; Europe/en) 1 0% 20.02.2007 03:58:56
yacy (www.yacy.net; v20040602; i386 Linux 2.6.16.20-vs2.0.2-rc22-lsgtux0; java 1.4.2-02; Europe/en) 1 0% 20.02.2007 04:04:11
TsWeb/Nutch-0.8.1 (TSWeb; tswebtek.com; [email protected]) 1 0% 23.02.2007 13:49:19
yacy (www.yacy.net; v20040602; i386 Linux 2.6.15-27-386; java 1.4.2-02; Pacific/de) 1 0% 27.02.2007 03:37:47
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2 1 0% 02.03.2007 16:11:14
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_11; Europe/de) 1 0% 02.03.2007 19:38:34
FDF-Bot - Free Domainfinder by www.geld13.de 1 0% 05.03.2007 16:25:52
DataparkSearch/4.43 (+http://dataparksearch.org/bot) 1 0% 11.03.2007 13:11:08
shaboyi spider 1 0% 13.03.2007 09:29:38
yacy (www.yacy.net; v20040602; i386 Linux 2.6.21-rc2; java 1.5.0_09; Europe/de) 1 0% 16.03.2007 09:43:00
Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com) 1 0% 20.03.2007 12:53:52
larbin_2.6.1 ([email protected]) 1 0% 21.03.2007 00:36:18
sogou web spider http://www.sogou.com/docs/help/webmasters.htm#07 1 0% 28.03.2007 17:42:07
yacy (www.yacy.net; v20040602; i386 Linux 2.6.17-11-generic; java 1.5.0_07; Europe/en) 1 0% 31.03.2007 11:01:43
yacy (www.yacy.net; v20040602; i386 Linux 2.4.34; java 1.6.0; Europe/en) 1 0% 01.04.2007 22:26:37
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.8.1 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; [email protected]) 1 0% 06.04.2007 04:39:06
locallytargetbot/locallybotV2 (locallybotV2; http://www.locallytarget.com; [email protected]) 1 0% 12.04.2007 07:09:02
tellbaby/Nutch-1.0-dev (http://www.tellbaby.com) 1 0% 17.04.2007 20:20:48
LocalcomBot/1.3.0 (+http://www.local.com/bot.htm) 1 0% 18.04.2007 00:36:19
Java/1.6.0_01 1 0% 19.04.2007 12:33:51
yacy (www.yacy.net; v20040602; i386 Linux 2.6.18-4-686; java 1.5.0_10; GMT+01:00/de) 1 0% 26.04.2007 12:58:49
yacy (www.yacy.net; v20040602; i386 Linux 2.6.14.2-rootserver; java 1.4.2-03; Europe/en) 1 0% 02.05.2007 21:10:39
yacy (www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.6.0_01; Europe/en) 1 0% 02.05.2007 21:11:43
yacy (www.yacy.net; v20040602; i386 Linux 2.6.15-1-686; java 1.5.0_04; Europe/en) 1 0% 02.05.2007 21:11:47
yacy (www.yacy.net; v20040602; i386 Linux 2.6.18-4-vserver-k7; java 1.5.0_10; Europe/en) 1 0% 03.05.2007 08:28:06
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; InfoPath.1; MSIECrawler) 1 0% 03.05.2007 14:56:44
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; DigExt) 1 0% 05.05.2007 05:49:22
Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org) 1 0% 05.05.2007 18:45:07
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR 1.1.4322; .NET CLR 1.0.3705) 1 0% 05.05.2007 20:09:59
yacy (www.yacy.net; v20040602; i386 Linux 2.6.8-24.25-smp; java 1.4.2_11; Europe/de) 1 0% 05.05.2007 23:43:02
Crawler0.1 1 0% 06.05.2007 12:01:42
yacy (www.yacy.net; v20040602; i386 Linux 2.6.16.13-4-smp; java 1.6.0_01; Europe/de) 1 0% 10.05.2007 02:38:09
my-heritrix-crawler(+http://mywebsite.com) 1 0% 10.05.2007 23:44:10
SygolBot http://www.sygol.com 1 0% 11.05.2007 03:22:21
iSEEKbot/iSEEKbot-0.9-dev (http://beta.iseek.com/iseekbot.html; bot at iseek dot com) 1 0% 13.05.2007 06:36:48
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Arcor 5.006; SU 3.002; MSIECrawler) 1 0% 17.05.2007 17:36:12
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 1&1 Internet AG by USt; MSIECrawler) 1 0% 17.05.2007 17:42:15
Blaiz-Bee/2.00.8315 (BE Internet Search Engine http://www.rawgrunt.com) 1 0% 21.05.2007 20:10:24
yacy (www.yacy.net; v20040602; i386 Linux 2.6.18; java 1.5.0_06; GMT+01:00/en) 1 0% 23.05.2007 14:33:49
ydeywjqggarmtah dj swmayouiqc7 S 1 0% 24.05.2007 07:23:36
Jakarta Commons-HttpClient/3.0.1 1 0% 27.05.2007 02:21:38
wectar/Nutch-0.9 (nectar extracted form the glorious web; http://goosebumps4all.net/wectar; see website) 1 0% 29.05.2007 02:15:47
yacy (www.yacy.net; v20040602; i386 Linux 2.6.20-16-generic; java 1.4.2-02; GMT+02:00/fi) 1 0% 31.05.2007 22:37:25
flatlandbot/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected]) 1 0% 02.06.2007 23:25:01
Mozilla/5.0 (compatible; MileNSbot/0.1beta; http://www.milens.net/robotstxt.html) 1 0% 04.06.2007 13:50:25
Mozilla/4.0 (compatible; bogospider; Linux) 1 0% 09.06.2007 08:46:09
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; ZangoToolbar 4.8.3; HbTools 4.8.4; MSIECrawler) 1 0% 12.06.2007 19:46:52
yodaoice 1 0% 16.06.2007 11:29:43
gsa-crawler-upgrade-config (Enterprise; C4-JN7MLGG2WLJAA;) 1 0% 17.06.2007 07:34:38
Mozilla/5.0 (compatible; heritrix/1.12.1 +http://newtestbabes.com) 1 0% 21.06.2007 07:18:48
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; +http://newb.com) Gecko/20070508 Firefox/1.5.0.12 1 0% 22.06.2007 09:08:22
Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/419 (KHTML, like Gecko) Safari/419.3 1 0% 26.06.2007 13:35:37
Lynx/2.8rel.3 libwww-FM/2.14 1 0% 30.06.2007 19:18:03
Mozilla/5.0 (compatible; heritrix/1.12.0 +http://aleph-webservices.com) 1 0% 03.07.2007 23:34:38
yacy (www.yacy.net; v20040602; i386 Linux 2.6.9-023stab043.1-smp; java 1.5.0_03; Europe/en) 1 0% 04.07.2007 00:52:12
WebAlta Crawler/1.3.34 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU) 1 0% 04.07.2007 12:14:48
User-Agent: NokiaN70/. FASTMobileCrawl/6.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 1 0% 07.07.2007 13:07:28
NutchCVS/Nutch-0.9 (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected]) 1 0% 08.07.2007 10:07:19
DataparkSearch/4.46 (+http://dataparksearch.org/bot) 1 0% 14.07.2007 01:19:26
xxyyzz 1 0% 14.07.2007 03:04:52
infomisa/Nutch-0.9 1 0% 19.07.2007 15:51:28
Spock Crawler (http://www.spock.com/crawler) 1 0% 22.07.2007 02:04:29
abc/Nutch-0.9-dev (abc; http://abc#11.us; abc at abc dot com) 1 0% 23.07.2007 03:17:35
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30) 1 0% 23.07.2007 11:25:22
yacy (www.yacy.net; v20040602; x86 Windows 2003 5.2; java 1.6.0_02; Europe/de) 1 0% 24.07.2007 04:12:17
ActiveAthleteCrawl/0.7.2 (ActiveAthlete Search Bot; http://www.activeathletesearch.com/; [email protected]) 1 0% 26.07.2007 00:19:28
Mozilla/4.0 (compatible; MSIE enviable; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea; +http://ws.daum.net/aboutkr.html) 1 0% 27.07.2007 15:37:56
http://www.information-online.com/ 1 0% 31.07.2007 03:37:17
NextGenSearchBot 1 (for information visit http://about.zoominfo.com/About/NextGenSearchBot.aspx) 1 0% 02.08.2007 00:01:22
Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) 1 0% 12.08.2007 10:33:55
WannesWeb-Group Agent/Nutch-0.9 (Agent de recherche pour le moteur WannesWeb-Group; www.wannesweb-Greoup.com; [email protected]) 1 0% 15.08.2007 03:22:29
FuseBulb.Com 1 0% 20.08.2007 08:15:11
Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.8.1.6) Gecko/20070723 Firefox/2.0.0.1 (Windows) 1 0% 20.08.2007 18:14:44
Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.accelobot.com) 1 0% 23.08.2007 04:07:38
Wget/1.10.2 (Red Hat modified) 1 0% 25.08.2007 06:09:46
softgardenspider/Nutch-0.9 (softgarden search engine; www.softgarden.de/search; search at softgarden dot de) 1 0% 25.08.2007 17:24:22
woriobot (+http://www.worio.com) 1 0% 28.08.2007 19:24:50
yacy (www.yacy.net; v20040602; x86 SunOS 5.11; java 1.6.0_01; Europe/de) 1 0% 01.09.2007 21:17:34
yacy (www.yacy.net; v20040602; amd64 Linux 2.6.20-16-generic; java 1.5.0_11; GMT+01:00/en) 1 0% 02.09.2007 18:29:07
Robotgenius crawler/Nutch-0.9 (http://robotgenius.net; misc at robotgenius dot net) 1 0% 03.09.2007 14:22:54
testnutch/Nutch-1.0-dev (testnutcher; http://private; [email protected]) 1 0% 03.09.2007 21:58:26
msnbot/1.0 (+http://search.msn.com/msnbot.htm 1 0% 04.09.2007 00:59:01
qq5mpk5quwhyjxlyairiagg5j 1 0% 06.09.2007 12:57:07
Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot) 1 0% 06.09.2007 16:42:53
SeznamBot/2.0 (+http://fulltext.seznam.cz/) 1 0% 07.09.2007 15:38:02
Java/1.5.0_01 1 0% 08.09.2007 10:16:08
nrsbot/5.0(loopimprovements.com/robot.html) 1 0% 13.09.2007 06:34:11
sait/Nutch-0.9 (SAIT TEST AGENT; http://www.sait.samsung.co.kr) 1 0% 15.09.2007 04:43:51
IndyLabs_Marius_Crawler ([email protected]) 1 0% 15.09.2007 09:23:49
Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net) 1 0% 15.09.2007 16:26:42
Sogou Pic Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07) 1 0% 19.09.2007 18:58:26
yacy (www.yacy.net; v20040602; i386 Linux 2.6.22-ARCH; java 1.6.0_02; Europe/de) 1 0% 22.09.2007 00:59:12
Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.greaterera.com/) 1 0% 23.09.2007 15:18:23
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 ([email protected]) 1 0% 24.09.2007 08:15:53
Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) [email:[email protected]] 1 0% 24.09.2007 11:53:26
LinkWalker/2.0 1 0% 25.09.2007 14:15:50
Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1) 1 0% 25.09.2007 23:03:24
disco/Nutch-0.9 (experimental crawler; www.discoveryengine.com; [email protected]) 1 0% 26.09.2007 07:30:53
ASAHA Search Engine Turkey V.001 (http://www.asaha.com/) 1 0% 30.09.2007 13:55:47
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Seekmo 10.0.341.0; MSIECrawler) 1 0% 06.10.2007 09:57:09
owsBot/0.2 (owsBot; www.oneworldstreet.com; owsBot) 1 0% 06.10.2007 10:34:25
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) ([email protected]) 1 0% 09.10.2007 13:13:21
yacy (www.yacy.net; v20040602; i386 Linux 2.6.20-16-generic; java 1.6.0; GMT+01:00/de) 1 0% 10.10.2007 13:23:01
gonzo1[P] +http://www.suchen.de/faq.html 1 0% 14.10.2007 10:28:51
BCNutch/Nutch-0.9 (BrightCloud crawler; http://www.brightcloud.com/crawler.asp) 1 0% 15.10.2007 14:30:41
testBOT/0.002 1 0% 16.10.2007 08:22:43
MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default) 1 0% 21.10.2007 05:37:03
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12; ips-agent) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7 1 0% 22.10.2007 06:43:48
Mozilla/5.0 (compatible; EARTHCOM/2.2; +http://enter4u.eu) 1 0% 23.10.2007 01:31:47
nutch/Nutch-0.9 (+http://www.cityreview.org/crawler/) 1 0% 26.10.2007 08:29:52
LiveTrans/Nutch-0.9 (maintainer: cobain at iis dot sinica dot edu dot tw; http://wkd.iis.sinica.edu.tw/LiveTrans/) 1 0% 27.10.2007 06:11:11
Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.yodao.com/help/webmaster/spi der/;) 1 0% 27.10.2007 17:01:19
nestReader/0.2 (discovery; http://echonest.com/reader.shtml; reader at echonest.com) 1 0% 29.10.2007 17:18:52
Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Win 9x 4.90; QXW03406; .NET CLR 1.1.4322; MSIECrawler) 1 0% 30.10.2007 17:28:00
yacy (www.yacy.net; v20040602; x86 Windows 2003 5.2; java 1.6.0_03; Europe/de) 1 0% 31.10.2007 19:47:06
netseer/Nutch-0.9 (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; [email protected]) 1 0% 01.11.2007 03:13:17
yacy (www.yacy.net; v20040602; ppc Linux 2.6.22-2-powerpc; java 1.5.0; Europe/de) 1 0% 01.11.2007 06:44:09
Mozilla/5.0 (Windows;) NimbleCrawler 2.0.2 obeys UserAgent NimbleCrawler For problems contact: [email protected] 1 0% 01.11.2007 17:13:28
Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9 1 0% 06.11.2007 16:18:23
AOL_DAC_Search_Spider/Nutch-0.9 (Demo Thing; [email protected]) 1 0% 07.11.2007 10:02:49
LWP::Simple/5.805 1 0% 08.11.2007 02:07:20
AOL_Daniel_Clark_Spider/Nutch-0.9 (AOL Search; [email protected]) 1 0% 09.11.2007 02:49:09
Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.netseer.com/crawler.html; [email protected]) 1 0% 09.11.2007 07:59:47
nutch/Nutch-1.0-dev (nutch.biz; [email protected]) 1 0% 09.11.2007 16:22:03