MSN spider crawling my site at crazy rate

bpeh · 02-06-2006, 12:11 AM

hi,

i noticed an increase in msnbot activity in the past few days? is it only me or anyone else out there? googlebot used to be no.1, now msn overtook the bandwidth used by more than 2 times.

something is happening, not sure what.

kenni · 02-06-2006, 12:51 AM

I never noticed this, come to think about it, i never checked. If it is causing a problem, you can always block off the bot

microdude431 · 02-06-2006, 12:53 AM

How much is it using daily?

compuXP · 02-06-2006, 02:55 AM

Actually I read somewhere that robots.txt file doesn't work against MSN bot because Micro$oft wants as many indexes as possible...

microdude431 · 02-06-2006, 03:52 AM

Quote:

Originally Posted by compuXP

Actually I read somewhere that robots.txt file doesn't work against MSN bot because Micro$oft wants as many indexes as possible...

Good for me, since my site is getting good ranking for its keywords.

a1ds · 06-19-2006, 06:42 PM

bpeh

Which site are talking about from your signature?

loanskey · 06-19-2006, 06:44 PM

MSN is fastest in crawling. But it donot crawl deep as google do.
MSN is also good for new websites with less PR

02-06-2006, 12:11 AM	#1
bpeh Member Join Date: Feb 2006 Location: Australia Posts: 41	MSN spider crawling my site at crazy rate hi, i noticed an increase in msnbot activity in the past few days? is it only me or anyone else out there? googlebot used to be no.1, now msn overtook the bandwidth used by more than 2 times. something is happening, not sure what. __________________ Website Reviews, Web Design Ideas, Internet Articles, Directory

02-06-2006, 12:51 AM	#2
kenni Registered Member Join Date: Dec 2005 Posts: 1,543	Re: MSN spider crawling my site at crazy rate I never noticed this, come to think about it, i never checked. If it is causing a problem, you can always block off the bot __________________ Wanna thank someone? Give 'em a rep. More info.

02-06-2006, 12:53 AM	#3
microdude431 Senior Member Join Date: Dec 2005 Posts: 349	Re: MSN spider crawling my site at crazy rate How much is it using daily? __________________ Free Webmaster Directory

02-06-2006, 02:55 AM	#4
compuXP Retired Member Join Date: Dec 2005 Posts: 139	Re: MSN spider crawling my site at crazy rate Actually I read somewhere that robots.txt file doesn't work against MSN bot because Micro$oft wants as many indexes as possible... __________________ Retired Moderator/Member/Friend/Helper - it was great working with all of you!

06-19-2006, 06:42 PM	#6
a1ds Member Join Date: Feb 2006 Posts: 137	Re: MSN spider crawling my site at crazy rate bpeh Which site are talking about from your signature? __________________ 6 Deep Links Bid Links Directory - A1 Directory Search.com

06-19-2006, 06:44 PM	#7
loanskey Smurf Join Date: Mar 2006 Posts: 23	Re: MSN spider crawling my site at crazy rate MSN is fastest in crawling. But it donot crawl deep as google do. MSN is also good for new websites with less PR __________________ ZillR : Indian social Network

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
I am having problems using Spider View	keiscokr	Help and Support	4	11-01-2006 02:06 AM
Search Engine Spider Simulator : Testers Wanted	fnsbe	Tools & Software	1	10-16-2006 09:31 AM
Preventing google from crawling some folders	loanskey	Google	3	10-15-2006 07:42 PM
Crazy Domain name what to do now?	AMH	Domain Name	4	03-30-2006 08:02 AM