SIGIA-L Mail Archives: Re: [Sigia-l] Don't submit websites to search engines?
Re: [Sigia-l] Don't submit websites to search engines?
As the author of the Deep Web article, I just want to respond to a couple of
Ziya's points:
>> Those of us who place our faith in the Googlebot may be surprised to
>> learn that the big search engines crawl less than 1 percent of the
>> known Web.
> Is there any concrete evidence of this?
Yes. In February, Google announced plans to expand its searchable index to
about 6 billion documents; reputable sources like Yahoo! and Brightplanet
have pegged the size of the Deep Web at upwards of 100 billion documents.
> Cut and paste "The Chemical and Biological Warfare Threat" into Google,
> elapsed time less than 0.28 seconds:
>
> http://www.sci.sdsu.edu/classes/biology/bio610/bernstein/PDFS/
> Dr.Sabbadini/warfare.pdf
The document Ziya refers to is not the CIA report I mentioned in my article;
it just happens to share the same title. This article was written by Dr.
Roger Sabbadini of the Institute for International Security and Conflict
Resolution at San Diego State University. The CIA report still does not
appear in Google (or Yahoo!) search results.
regards,
alex
-----------
alex wright
alex_at_agwright.com | www.agwright.com
------------
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments
Searchable list archive: http://www.info-arch.org/lists/sigia-l/
________________________________________
Sigia-l mailing list -- post to: Sigia-l_at_asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
This archive was generated by hypermail 2.1.6
: Tue May 18 2004 - 14:52:30 EDT
|