Reading Reflection: Usability Testing

We read two chapters on usability testing this week. The UX book refers usability testing as rigorous evaluation, while activities such as experts inspection and heuristic evaluation as rapid evaluation. Is the one called rigorous actually more rigorous or effective? This makes me think about Apple again. Sometimes, the experts’ experiences combined with their understanding of the users’ goals, and with proper marketing strategies can serve longer way in the product’s life. Understandably, the word “rigorous” here is in terms of the overall scientific procedures of the methods, yet throughout the text, it actually emphasizes a lot of the difference between the UX evaluation, and formal scientific research (esp. quantitative research). For example, (1) Quantitative UX data collection and analysis “is emphasized less than it used to be in previous usability engineering books because of less focus in practice on quantitative user performance measures and more emphasis on evaluation to reveal UX problems to be fixed.” (p.504) (2) When talking about participants recruiting, the term “sample” is not appropriate, “because what we are doing has nothing to do with the implied statistical relationships and constraints.” (p.511) (3) The authors spend a lot of text discussing the proper number of participants, saying it ultimately depends on the specific cases, and the UX researchers’ experience and intuitions. In a word, UX evaluation is very practical activities focusing on fixing the real problems.

Other things I wish to remember for preparing the UX evaluation: 

1. It is very important to set the goal, and define the scope of the evaluation. “Do not let short-term savings undercut your larger investment in evaluation and in the whole process lifecycle.” (p.504) Choosing the evaluation methods and techniques should be goal-driven.

2. We should try to engage as many as people in the whole team in at least some evaluation. It is not only the UX team’s job to do the evaluation, other members of the project should be involved as much as possible. This can begets ownership, and is necessary for the UX evaluation results to be taken seriously and solve the problems.

3. When preparing the tasks for the evaluation, we need to allow time for the users to do free exploration, and we can also ask the users to define tasks.

Things I wish to remember for running the evaluation session: 

1. There are roughly four types of data: quantitative UX data (observed data and subjective data obtained mainly using questionnaires) , qualitative UX data (mainly, critical incidents collection), emotional impact data, and phenomenological data (how the products integrate into the users’ lifestyle, mainly collected using variations of diary-based methods).

2. The UX book mentions the HUMAINE project on emotional impact research. I went ahead and checked the report, and would like to share here: The HUMAINE project website. The HUMAINE report on taxonomy of affective system usability testing.



MyPurdue “Look Up Classes” Usability Inspection

The web interface of myPurdue has lots of problems, but the one bothers me the most is the course navigation, since I use it quite often to look for courses. When I log in to myPurdue, and click the “Academic” tab, and then click the “Look Up Classes” at the navigation bar on the left, the interface of Pic. 1 shows up. The issue on this page is that the “Reset” button is completely useless, since I can just switch among different terms using the drop-down menu or choose “none” to reset, though I don’t think anybody looking for courses would choose “none”. This is okay, since I do not use it, it doesn’t bother me much.


Then I choose the term, and click “Submit”. Pic. 2 is what shows next. The two buttons at the bottom are confusing, since we usually see “Search” and “Advanced Search” elsewhere on the web, rather than “Course Search” and “Advanced Search”. Based on the instruction on top, “Course Search” is to search by subject, while “Advanced Search” is to search by section. Here, the order of concepts is that each “subject” contains a series of “sections”, while each “section” contains many individual courses. When you click “Course Search”, it goes to the sections under the subject you choose. When you click “Advanced Search”, it would go to the list of individual courses after your specification. So “subject” means “major” here, which is really confusing for me. I thought subject means an individual course!  Put all these aside, the subject list on this page actually only applies to “Course Search”, not to “Advanced Search”. The two buttons on the bottom are a bit far away from the subject list window, so my laptop screen usually cannot display all them at once. So I would only see the subject list window at the beginning, then I would start to choose the subject. The list of subjects is kind of long. After I spend a long time scrolling down and choose the subject, and then realize I can click “Advanced Search”. After I click “Advanced Search”, a new interface shows up, my choice of subject is gone, it asks me to choose from the subjects again! Maybe myPurdue should move the two buttons to the top of the subject list window and also make other accommodations to make all such buttons consistent.

Pic. 2

If I click “Course Search” button, it will take me to a series of sections shown in Pic. 3. Here, I don’t have a button to go back and start a new search like the one in Pic. 4. I have to choose a section to view, unless I go back to the Academic tab.

Pic. 3

Pic. 4

Pic. 5 is what it shows when I go to “Advanced Search”. The instructor list here is really really long, and I never have the patience to use it. If I have a specific professor name I would like to search for, the system should just let me input the name to search. For now, I’d rather go to the entire course list, and use Ctrl+F or Command+F (on Mac) to search for the professor’s name in the broswer, which is much faster than this humongous name list.

Pic. 5

Future Interface for Researchers: NUI+BCI

I’m a researcher. I read and write all the time. When I write, I need to cite what I’ve read, in order to identify the holes in what others have written and provide support for my own argument.

I read various publications: academic papers, books, reports, magazines, blogs, etc. I read from various media: either printed or electronic formats from my own laptop, or my old PC, or university lab computers, or my iPhone, or maybe iPad or Kindle etc. I read in various places and position: at the dining table while eating breakfast, at my study desk at home, in the office, in the library, in a cafe, sitting on the floor on my Yoga mat, or lying in bed.

I try to be organized of my reading, so that I can remember to cite them later while writing. I make folders and piles. I highlight interesting and important sentences while reading. I take notes. I use citation management tools. All these and a good memory have made my life in graduate school a tiny little bit easier, but I still find it is very difficult to keep track of what I’ve read and remember to cite them later if I am short on time and have an intense writing task and urgent deadline. While acknowledged that how to manage and organize reading and writing is a very very important skill to learn in graduate school, I still wish there would be some innovative methods to ease this process.

I had this idea while I was doing my PhD readiness assessment earlier this semester, in which I had to read and write a lot in two weeks. At some moments, I was so exhausted. I had piles of printed papers, books, notebooks, and laptops (with even more papers and notes inside) lying around my room on the desk, on the floor, and on the bed. When I read, I saw many interesting quotes that I might cite later, but I didn’t have the time to jot all them down, because I might only cite a small portion of them, jotting all of them down would take too much time out of my precious two weeks. But later, when I really wanted to cite one of them, it actually took some time to find it, or I just simply couldn’t remember and gave up. I wish at that time, I could have the ability to just crab the important sentences out of the papers or the computer screen along with the descriptors where they came from using a simple hand gesture (the key point here is that, no matter what kind of media or formats I am reading, I have to be able to crab them using a simple hand gesture), and throw them on the surface of the wall or a big canvas somewhere. Then I could arrange and select them just by eyeing them or thinking about them, because I was too tired lying in bed and couldn’t stand up and touch the wall. (I may also settle with waving hands and kicking legs if eyeing and thinking are still too difficult to implement for now:) Then maybe I could think about how I would write the argument, and the computer can just write it down for me.

I think the key here is how to integrate all different forms of media, how to integrate the physical world, our body and brain, and the digital world. How to smooth the connections and communications among all them. We can pretty much manage one device or one type of task very well, but when there are so many different forms of stuff, we get dis-organized and lost. I think we are actually close to realizing this integration, there are cool things done by the natural user interface (NUI) and brain computer interface (BCI) communities. The MIT media lab is among the famous ones. I do believe the future of HCI lies in the combination of NUI and BCI to solve this integration problem.

One side thought here is that context does matter a lot for design. I had this idea when I was working at home. So I wanted to be able to wave and kick and be able to write while lying in bed. But later I moved my “battle field” to the office on campus, then my urge to wave and kick stopped since it would be inappropriate in the office. So I am essentially developing a persona of a researcher based on myself and think about the context and need of this persona.

RAA3: Analyzing #insomnia on Twitter

Jamison-Powell, S., Linehan, C., Daley, L., Garbett, A., & Lawson, S. (2012). “I can’t get no sleep”: Discussing #insomnia on Twitter. Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (pp. 1501–1510). Retrieved from
Purpose of the Research: 
(1) To explore the role of social media in the discussion of mental health issues, and with particular reference to insomnia and sleep disorders: whether individuals are disclosing their insomnia related information online, if so, what they are disclosing.
(2) To provide analytical results to those in the interaction design community interested in integrating online social media systems in health interventions.
A mixed-method approach is used to analyze 18,901 tweets using hashtag #insomnia. First, an automatic content analysis is used to confirm “whether individuals are disclosing their insomnia related information online”. Second, a qualitative thematic analysis is performed to answer the question “what individuals are disclosing about their insomnia”.
(1) Content analysis using a software named Linguistic Inquiry and Word Count (LIWC) . Simply put, this software contains a pre-defined English dictionary which includes categories of negative words and positive words, as well as word roots related to health, time, anger, anxiety, sadness, etc. The software performs frequency counting and statistical analysis of the input text based on this dictionary. In order to compare these #insomnia tweets with the general posts on Twitter, 17,532 random general non-specific tweets on Twitter were used as a control group. LIWC is used to compare the #insomnia tweets and the non-specific tweets, and produces table 1 below.
(2) Inductive thematic analysis. The authors acknowledge that the LIWC software analysis results maybe inaccurate because of the subjectivity and ambiguity of  informal human language, as well as the large amount of abbreviations used in tweets due to the character limit. So they perform inductive thematic analysis. A sample of 749 is taken from every 25th tweet of the original 18,901 tweets. One researcher first read the 749 tweets for several times, and comes up with 49 categories, and gives the results to two other researchers. The three of them discuss and decide on 45 categories. Then they take another sample of 514 tweets from every 35th tweet of the original 18,901 tweets, and code the 514 tweets into the 45 categories independently.  Inter-rater reliability is also examined.  All the researchers finally refine the categories into a hierarchy in figure 1.
Main Findings: 
(1) Results in table 1 below confirms that individuals do disclose insomnia related information online.
(2) Inductive thematic analysis categorize the content of #insomnia tweets into two major categories: describing the insomnia experience and coping with insomnia.
The authors claim that this paper, in a generic sense, contributes to a growing body of literature regarding online self-disclosure. Based on the analysis results, they come up with a brief design requirement for a social network structure allowing insomnia therapy to be delivered in an online group. “The structure would be a useful format for users to exchange support and practical information. To allow users to exercise catharsis,  a space could be provided to disclose symptoms and frustrations together with an area where the users can “communicate with” insomnia. To comply with the principles of sleep hygiene, a digital barrier should be in place in any interactive platform, this would reduce the amount of access a user has to an online support network at times when they should be refraining from stimulating environments. However, rather than simply restrict all access a “life-line” should be kept open, providing information on good sleep hygiene and progressive muscle relaxation exercises to aid sleep. “
This paper relates to what we learned in CGT512 how to analysis qualitative data and thus produce design requirements.
This paper also relates to my research interests in social media analytics for instrumenting the social and emotional aspects of college students’ life. It provides information on useful software and methods.

Reading Reflection: Information Organization and Prototype

Information Architecture Reading from Brinck, Gergle, & Wood :

1. What is information architecture?

Information architecture refers to the structure or organization of the website, especially how each page relates to each other. This involves content analysis and planning. Information architecture is rooted from database design and information retrieval, and strongly influenced by HCI, library science, and  the psychology of how human beings navigate and organize information.

2. What are some user-centered techniques for creating the information architecture?

Information architecture can be created based on the strict logic of the content, but this is rarely the case. Most of the time, there is no single logical organization of the content, even if there is, this may not be the best for the users. Because users are not always completely logical, neither are they always very clear about their specific goals of navigating the website. This reminds me the MEMEX machine by Vannevar Bush we discussed at the beginning of this semester which reminds us that human beings organize information based on associations rather than hierarchy. User-centered information architecture design takes human beings’ perception and cognition into consideration. There are roughly top-town and bottom-up approaches to create information architecture. When designers encounter uncertainty in information architecture design, a useful method is to let the users do cart sorting activities.

3. What are various website topologies (ways of organizing information)?

The topology is the primary way that pages are linked together. The most popular one is hierarchy or tree. Other website topologies include linear, matrix (or Grid), full mesh, arbitrary network, and hybrid topology. It is important to consider the breadth (7) and depth (3) of the hierarchy. Typically, the breadth of the tree should be less than seven branches (though the authors of this book do not think this number is justified), and the depth of the tree should not exceed three levels. Semantics is a task-based and more user-friendly method of organizing information.

The Two Research Articles on Information Organization:

What navigation topologies and structures are better in what kind of situations?

How users navigate information depends on whether they are clear about what they are looking for. When the task is more specific, the users tend to use the traditional search interface; when the task is more general, tag cloud is preferred. Usage-oriented hierarchy is preferred compared with subject-oriented hierarchy under all circumstances. To sum it up, how users navigate information depends on their tasks, so that the design of information architecture should match all levels of users’ tasks.

UX Book Chapter 11 On Prototyping:

1. How do you choose between the different types and fidelity levels of prototypes?

How to choose the fidelity levels of the prototypes depends on the stage of progress of the overall project and the design perspective that is prototyped. The UX book mentions the horizontal and vertical prototyping. I feel I’m not very clear about the differences between features and functionality. What is feature? What is functionality? Why feature is horizontal, while functionality is vertical?

2. What are some pitfalls to avoid when prototyping?

Prototypes need to match the perspectives of the products. Viewers of the prototypes need to be chosen carefully. Low-fidelity prototyping needs to be carefully explained to the viewers if they are outside of the design team. Get organizational understanding that prototyping can reduce risks both on software engineering and the UX sides. Be honest about limitations of the prototyping and do not overwork the prototypes.

Engineering and Beauty

We discussed the three paradigms of design in a previous class: the engineering paradigm, the human information processing (HIP) paradigm, and the phenomenological paradigm. The engineering paradigm stresses that things should work, the HIP paradigm stresses that things should be easy to use, while the phenomenological paradigm stresses that things should be pleasant to use. I think the three paradigms are along the line of evolution, and the ultimate pursuit of the humankind is about aesthetics and beauty as consistent with the phenomenological paradigm. When we talk about design for the users’ life goal, I think the kind of users’ life goals as related to product development are mostly not very specific, not about that they want to retire with kids and family. Rather, it is a more abstract goal of all humankind—we all want to be identified with something beautiful, because beauty is what touches the human mind. I think this is why Apple products succeed in the engineering world. This interesting blog post reminds engineers to think about the benefit of beauty. 

Crop vs. Mask

As much as I like the iWork on the Mac especially Keynote compared with Powerpoint and Word in MS Office. I still have not got used to the “mask” function. “Mask” basically does the same thing as “Crop” in MS Office, but it provides more flexibility: both the mask window and the picture can be moved and  adjusted in size. However, I do not feel this flexibility adds to any essential value. It still does the same thing as “crop”, and nothing more (at least in my 1 year and 4 months usage experience with Keynote, I have not needed to use anything more out of “mask” than just “crop”). It only adds to the hecticness of the users: sometimes I want to move the whole picture including the mask, but it turns out I only moved the under layer picture, or I want to adjust the mask but instead adjusted the picture. I have to spend effort to think whether I am adjusting or moving the picture, the mask or the entire thing. I am not very sure what kind of perceptual, cognitive, or psychological principles exactly apply here, or is it just me? At least, too much flexibility do not really add to the pleasantness of the usage experiences. I feel “mask” and “crop” are two different metaphors for the same thing, one is to cover the part you do not want, the other is to cut the part you do not want, and cutting maybe more direct than covering.

Moreover, I vaguely remember a previous version of MS Office had functions to do round-edged or circle cropping, but I cannot find this function anymore, or maybe it is just hidden somewhere. I find that very useful when you try to design a good-looking slides with round-edged pictures, but I don’t know why they decided to take this function down or hide it.

How do you feel about “mask” and “crop”? Which metaphor do you prefer?