SIGIA-L Mail Archives: RE: [Sigia-l] ROI/Value of Search Engine
RE: [Sigia-l] ROI/Value of Search Engine Design - Resources?
From: Jared M. Spool (jspool_at_uie.com)
Date: Mon Feb 10 2003 - 23:02:22 EST
Boniface Lau wrote:
> > I come from a school that there isn't ever a problem with users --
> > it's always a problem with the design.
>Extreme statement like that is good for generating debates. I can even
>see academic holding on to such belief. But seeing it coming from
>someone who is supposed to have industry experience is a bit
>surprising, to say the least. Are you making the above statement just
>to be provocative?
As I said, I have this belief. I do not say it *just* to be provocative. It
is what I believe. I probably have a lot of surprising beliefs. It is not
my goal to partake in a public dissection of my beliefs, particularly on
this list which doesn't have a recent history of being a 'friendly, safe
place' for people to share their views. So, we're just going to leave this
conversation at this. I know you can respect that.
>It is also possible that the data have been mis-interpreted. Care to
>publish details of the study behind the UIE article "Why Amazon
>Succeeds -- And Why It Won't Help You"
>Regarding the UIE article "Why On-Site Searching Stinks"
>(http://www.uie.com/searchar.htm), I am sure many people would like to
>see the details you have for coming to the following extreme
>WOSSS> Our data showed that today's on-site search engines are worse
>WOSSS> than nothing -- significantly worse.
>Will you publish the study details to support the above extreme
Because I have a substantial research budget to fund, I have no problem
telling you that the study details (and many more like them) are available
to clients who pay to have access to that information. Feel free to contact
me if you'd like pricing information.
> > I question each of these things regularly and we're constantly
> > making improvements.
> > But, in this case, to the best of my knowledge and our research
> > abilities, I believe this user was the right user and their behavior
> > was indicative of what we often see when trying to perform similar
> > tasks.
>If you are so convinced, then may be the problem is indeed with
Yes it may. However, other institutions that have duplicated our results
have come to the same interpretation.
>For example, the UIE article "Why On-Site Searching Stinks" presented
>user interface problems and then claimed that "on-site search engines
>are worse than nothing". To me, that was data mis-interpretation.
>Would you mind explaining the logic behind the statement "on-site
>search engines are worse than nothing"?
The logic is simple: When users don't use on-site search, they are twice as
likely to find their content than when they do. This finding has been
duplicated several times by us and others.
It would be easy for you to duplicate it also. All you have to do is ask
users to find content (it works best with content they personally find
important instead of randomly assigned tasks) on a variety of sites. Track
their usage of categories versus search. Measure whether they actually get
to the content they were seeking. If your data matches ours, you'll see
about 30% with Search and about 53% without Search.
> > As we do more testing, we'll have more results, from which we'll
> > draw more inferences and opinions. We'll continue to present the
> > data so that everyone can draw their own inferences and opinions.
>Does that mean you will publish details of the studies behind the
>articles "Why Amazon Succeeds -- And Why It Won't Help You" and
>"Why On-Site Searching Stinks"?
It is available. Contact me for pricing. (I'm sorry, but it is not
economically feasible for us to make it available for free. We invest more
than $1 million annually in our research. We need to recoup our expenses
and continue to fund future research.)
Remember, if we haven't satisfied you that what we're saying is accurate or
believable, simply don't believe it. I can attest that the Sun will
continue to rise in the east every day, whether you agree with our findings
Our clients are happy with our results. They are making informed decisions
using our data. And they keep asking for more research. We must be doing
Jared M. Spool
User Interface Engineering
Don't miss User Interface 7 West, March 23-27, Burlingame, CA.
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments
ASIST IA 03 Summit: Making Connections
Searchable list archive: http://www.info-arch.org/lists/sigia-l/
Sigia-l mailing list -- post to: Sigia-l_at_asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
This archive was generated by hypermail 2.1.2
: Sun Nov 23 2003 - 22:55:36 EST