It looks like you're using an Ad Blocker.

Please white-list or disable AboveTopSecret.com in your ad-blocking tool.

Thank you.

 

Some features of ATS will be disabled while you continue to use an ad-blocker.

 

Google Robots VS Microsoft Robots.

page: 1
0

log in

join
share:

posted on May, 26 2005 @ 01:19 AM
link   
Ok this is.... well i dont know.

Why does the microsoft.com robots.txt file

www.microsoft.com...

Read like this





# Robots.txt file for www.microsoft.com...
#

User-agent: *
Disallow: /australia/careers/library/
Disallow: /backoffice/
Disallow: /canada/Library/mnp/2/aspx/
Disallow: /careers/international/
Disallow: /catalog/
Disallow: /communities/bin.aspx
Disallow: /communities/eventdetails.mspx
Disallow: /communities/blogs/PortalResults.mspx
Disallow: /communities/rss.aspx
Disallow: /downloads/info.aspx
Disallow: /france/formation/centres/planning.asp
Disallow: /france/mnp_utility.mspx
Disallow: /germany/library/images/mnp/
Disallow: /germany/mnp_utility.mspx
Disallow: /hwdev/
Disallow: /hwdq/
Disallow: /ie/ie40/
Disallow: /info/customerror.htm
Disallow: /info/smart404.asp
Disallow: /intl_kb/
Disallow: /intlkb/
Disallow: /isapi/
Disallow: /japan/enable/textview.asp
Disallow: /japan/hwdq/hwtest/
Disallow: /japan/mnp_utility.mspx
Disallow: /japan/products/library/search.asp
Disallow: /japan/showcase/print/default.aspx
Disallow: /japan/terminology/query.asp
Disallow: /library/errorpages/smarterror.aspx
Disallow: /library/toolbar/3.0/
Disallow: /mnp_utility.mspx
Disallow: /netherlands/mnp_utility.mspx
Disallow: /portugal/consumo/
Disallow: /proxy/
Disallow: /resources/casestudies/casestudyimageshow.asp
Disallow: /resources/casestudies/CompanyLogoShow.asp
Disallow: /resources/casestudies/ddi/companylogoshow.asp
Disallow: /resources/casestudies/ddi/showfile.asp
Disallow: /resources/casestudies/FindCaseStudyResults.aspx
Disallow: /resources/casestudies/showfile.asp
Disallow: /servers/
Disallow: /sna/
Disallow: /solutions/
Disallow: /technet/support/ee/
Disallow: /uk/mnp_utility.mspx
Disallow: /windows.netserver/
Disallow: /windows/catalog/
Disallow: /windows/embedded/
Disallow: /windows/powered/
Disallow: /windowsmobile/catalog/
Disallow: /winhec/


While the google robots.txt file

www.google.ca...

Read like this.




User-agent: *
Allow: /searchhistory/
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalog_list
Disallow: /news
Disallow: /nwshp
Disallow: /?
Disallow: /addurl/image?
Disallow: /pagead/
Disallow: /relpage/
Disallow: /sorry/
Disallow: /imgres
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
Disallow: /advanced_group_search
Disallow: /advanced_search
Disallow: /googlesite
Disallow: /preferences
Disallow: /setprefs
Disallow: /swr
Disallow: /url
Disallow: /wml
Disallow: /xhtml
Disallow: /imode
Disallow: /jsky
Disallow: /sprint_xhtml
Disallow: /sprint_wml
Disallow: /pqa
Disallow: /palm
Disallow: /hws
Disallow: /bsd?
Disallow: /linux?
Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /answers/search?q=
Disallow: /local?
Disallow: /local_url
Disallow: /froogle?
Disallow: /froogle_
Disallow: /print?
Disallow: /scholar?
Disallow: /complete
Disallow: /sponsoredlinks
Disallow: /videosearch?
Disallow: /videopreview?
Disallow: /videoprograminfo?
Disallow: /maps?
Disallow: /translate?
Disallow: /ie?


:w:

why does google disallow such things?

Meef.




 
0

log in

join