SIGIA-L Mail Archives: Re: [Sigia-l] Questioning common test sc
Re: [Sigia-l] Questioning common test scripting
From: InfoArchitect (InfoArchitect_at_ourbrisbane.com)
Date: Tue Jun 10 2003 - 04:06:27 EDT
> A positive message ("help us test X") works better than a negative
> one ("we are not testing you").
Exactly my point. From a cognitive psychology POV, using a negative
emphasis in this context is not acceptable, yet most of the scripts I
have come across (even some from cognitive engineering labs) contain
those words "We are not testing you". I'm just curious as to how
1. Started being used; and
2. Became a defacto standard without someone questioning the impact
such a statement may cause.
Using a redundant negative to emphasise such a point only serves to
create anxiety in the subject, and I can't see why it seems to be so
rife in the scripts that I have come across.
> How would you  know if the participant is at ease (or, for that
> matter, petrified, scornful, melancholic, adversarial, intimidated,
> aroused, etc.)?
>  As a non-medical person.
There are a number of methodologies for assessing emotional states in
cognitive psychology. In fact, Hudlicker (2000) cites that there are
over 200 anxiety instruments alone, many focusing on specific types of
anxieties (e.g., mathematics, social, etc.).
I actually tried to place a disclaimer in my original post so as not
to divert the discussion down this seemingly typical path of
tangential, semantic arguments that seems to occur. Obviously, I did
not cover all eventualities.
The point of this discussion is not how a professional can assess such
states, but why it seems to be common practise to use a script that I
hypothesise, would introduce a feeling of anxiety for participants,
when the object is to study typical responses that would occur in a
natural, relaxed environment.
I am wondering if the advent and subsequent popularity of ‘web
usability’, is diluting scientific rigor in human
factors/HCI/cognitive psychology evaluation techniques.
There has been much debate recently over the validity of data produced
by contemporary testing, but has anyone questioned:
The methods (not just if statatistical significance is practically
attainable, but are you using the correct type of test?);
The means (should this test be performed in a lab, in the field, etc.
Are the participants representative of a specified profile, or are
they simply "someone who uses the web"?); or
The qualifications/backgrounds of those performing the tests (do they
actually understand correct experimental rigor - know the difference
between an independent and dependent variable and the need to
eliminate confounding factors - or were they just taught how to carry
out some standard tests)?
User Experience Designer
Get your free email address from www.ourbrisbane.com
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments
Searchable list archive: http://www.info-arch.org/lists/sigia-l/
Sigia-l mailing list -- post to: Sigia-l_at_asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
This archive was generated by hypermail 2.1.2
: Sun Nov 23 2003 - 22:55:49 EST