Kath Straub at Human Factors Int’l releases a great list of Key Research Findings from 2002-3. It is one of the most useful design references I’ve seen recently. There’s a singular lack of research in this field – as opposed to trial-and-error – a problem compounded by the difficulty of finding and applying it. I hope to be as helpful as Ms. Straub with my IA Summit presentation, Incorporating Navigation Research into a Design Method, 28 Feb, Austin, Texas. Link courtesy PJB.
Category: Design
-
One-Question Survey
The One Number You Need to Grow by Frederick F. Reichheld is a great, short article on using one-question surveys that measure loyalty correlated with customer behavior. Highlights:
‘Every month, Enterprise polled its customers using just two simple questions, one about the quality of their rental experience and the other about the likelihood that they would rent from the company again. Because the process was so simple, it was fast. That allowed the company to publish ranked results for its 5,000 U.S. branches within days. …the company counted only the customers who gave the experience the highest possible rating…By concentrating solely on those most enthusiastic about their rental experience, the company could focus on a key driver of profitable growth: customers who not only return to rent again but also recommend Enterprise to their friends.’
‘In most of the industries that I studied, the percentage of customers who were enthusiastic enough to refer a friend or colleague — perhaps the strongest sign of customer loyalty — correlated directly with differences in growth rates among competitors.’
‘Companies have tended to focus on customer retention rates, but that measurement is merely the best of a mediocre lot…they basically track customer defections’
‘For a while, it seemed as though information technology would provide a means to accurately measure loyalty. Sophisticated customer-relationship-management systems promised to help firms track customer behavior in real time. But the successes thus far have been limited to select industries, such as credit cards or grocery stores, where purchases are so frequent that changes in customer loyalty can be quickly spotted and acted on. ‘ – Behavior is difficult to study and quantify, lots of data might help.
‘My personal bet for the top question (probably reflecting the focus of my research on employee loyalty in recent years) would have been "How strongly do you agree that [company X] deserves your loyalty?" Clearly, though, the abstract concept of loyalty was less compelling to customers than what may be the ultimate act of loyalty, a recommendation to a friend.’ – This question raises the same kind of emotion that occurs during real behavior, and might help explain why it’s a better indicator of behavior.
I like how he used a 10-point scale then clustered the results into three types of customers, avoiding “grade inflation.”
Also see the Loyalty Acid Test.
-
Mental Model vs. Innovation
So I bought an iPod used from a friend. Never has a device been so fulfilling and so annoying at the same time. I love the integration with the $.99-a-tune music store, but: the front panel is too sensitive, there’s no on/off switch, there’s no volume control (without attaching the additional wired remote do-dad), the wired remote do-dad doesn’t have any tactile feedback and doesn’t behave predictably – forcing one to refer back to the display, and so on.
I’m trying to love it, or at least make peace with it. Apple clearly produced an innovative product, and yet had to break several mental models to do so (one must hold the ‘play’ button to turn it off? Reminds me of my old Samsung mobile which one turns on by holding the ‘end’ key). I suppose designers must reach a point where it becomes difficult to do something better without doing it in a significantly different way (“Think…” (sorry)). Perhaps most people don’t notice, already being inundated with so much learning curve with every new device, but I’m a designer and I normally love Apple’s products so these quirks drive me insane.
-
Universal Principles of Design
Based on Adam’s writeup I picked Universal Principles of Design – a new book by Will Lidwell, Kritina Holden, and Jill Butler – down off the shelf and subsequently to the cash register. Adam’s take on it is accurate. Given my experience teaching – where people with undergraduate degrees in communication design from the Parsons School of Design still don’t know many of these principles – I’d say just about every novice to advanced designer could benefit from this book.
The book does a very good job at talking in simple ways that cut across design disciplines. It simultaneously shows the application of a principle to the Macintosh GUI and the Segway. Rare are the times when graphic, industrial, architectural, and software designers have overlapping areas of interest. The qualities of various sorts of rubber will keep industrial designers talking all night, and I couldn’t care less, and furthermore couldn’t keep up even if I did care. For this reason I’m curious to see where other efforts to unite these designers will go. One such effort is Interaction Designers, a group that strives to speak across these boundaries.
Another is the Interaction Design Institute Ivrea, and specifically the Hub, a community blog launched by the lovely Molly Steenson. Here Andrew sums up the difficult going in early posts as well as the great potential. The same sort of immediate sharing and the resulting learning that blogging has given many of us threatened to leave universities behind. Likewise, there are topics covered in these schools that aren’t disseminated to the rest of us, and this blog becomes a welcome conduit.
-
How Designers Follow Constraints
Notes on Web Site Designs: Influences of Designers Experience and Design Constraints (PDF) by Aline Chevalier and Melody Y. Ivory, which ‘demonstrates that the designers’ levels of expertise (novice and professional) as well as the design constraints that clients prescribe influences both the number and the nature of constraints designers articulate and respect in their web site designs.’ It’s part of the WebTango project at the iSchool, University of Washington.
‘We assert that understanding designers’ activities and identifying difficulties they encounter are essential to improving web site quality.’
‘we found studies showing that constraints are extremely important for understanding and for solving a design problem’
‘there is a wide gap between designers’ articulation of constraints and designers’ effective implementation of them.’
Those pesky designers! Seriously, reading this I feel like we could do a better job making constraints explicit in our personas and scenarios. Most I see are filled with a lot of nice details on our fictional character meant to make them more real but doesn’t add anything to the design process.
Also, I feel like we need something in-between the personas/scenarios and the design, an interaction model. More on this in a future post.
Even the experienced designers could only satisfy 75% of the constraints they were given. While they achieved up to 95% of the client constraints, they couldn’t satisfy more than half of the user constraints.
‘professional designers in the condition without constraints were able to infer client constraints, because they had contextual knowledge acquired through experience (stored as mental schemata)’
‘Results from the first two studies show that professional and novice designers encounter difficulties in effectively considering users’ needs during the design process, even though they focus mainly on users’ needs during the evaluation process.’
‘we argue that heuristic evaluation with ergonomic criteria suggested by Nielsen (2000) has not been adapted for web site designers (who have no human factors knowledge), because the ergonomic criteria are both too abstract and too numerous. Our hypothesis is that it would be more effective to provide designers with a subset of ergonomic constraints that respect the users’ real needs.’
Absolutely. There are simply too many guidelines to follow these days. There has to be a way of winnowing them down. Design patterns might help. A better design method might help.
Results: ‘1. Help novice designers to consider both user and client constraints. 2. Help professional designers to focus more so on user than client constraints or at least help them to strike a balance between the two actors. 3. Help designers, regardless of their levels of expertise, to consider and implement ergonomic constraints in their sketches.’
For the first two points, we suggest developing a knowledge-based system that fits the designer’s level of expertise (see Fischer et al., 1991). Specifically, the system should provide the following support:
- The system should help novice designers to identify constraints that need to be respected in the web site design.
- This system should also help novice designers to generate new constraints, through a design step oriented on the expectations of the client and the users. The system could help designers determine, based upon the current state of the design activity, additional information the designer may need to consider. For example, the system could propose questions for novice designers to ask the client.
- The system should help professional designers deal with a client who has many expectations, in particular, to help the designer consider more user constraints. For example, the system could suggest relevant constraints that the designer did not consider.
As solutions they suggest a focused questionnaire designers could use to evaluate designs through the design process, or an automated tool to evaluate the design. Both are probably helpful, but to truly advance I think we need to improve the method itself, not just devise better ways to find design flaws.
Thank you Aline and Melody.
-
Nielson’s Model of User’s Expertise
Plowing through research on navigation, just about everyone cites user expertise as a factor, regardless of the task being studied.
I’d like a better, more quantifiable, way to summarize a user’s level of expertise in a persona. This could lead to generalizations about what types of interaction will work for certain types of user.
Jakob Nielson, in Usability Engineering (as cited here), divides computer users into six categories along three dimensions based on the user’s experience: users with minimal computer experience and users with extensive computer experience for the dimension of knowledge about computers in general; novice users and expert users for the dimension of expertise in using the specific system; and user ignorant about the domain and users knowledgeable about the domain for the dimension of understanding of the task domain.
Thanks Jakob.
-
What Do Web Users Do?
Notes on What Do Web Users Do? An Empirical Analysis of Web Use (PDF) by Andy Cockburn and Bruce McKenzie, University of Canterbury, New Zealand. It was published in 2000, meaning the work was done earlier, but I still found the results useful.
They looked at the title, URL and time of each page visit, how often they visited each page, how long they spent at each page, the growth and con- tent of bookmark collections, as well as a variety of other aspects of user interaction with the web.
They only looked on 17 people, but gathered a lot of data on them. Netscape v4.x browsers.
Page views per person: The mean daily page visit count was approximately 42 pages for each user per day… …earlier studies… had approximate daily visit count means of fourteen (Catledge & Pitkow 1995) and twenty one (Tauscher & Greenberg 1997).
How often they revisited pages: Previous studies have shown that revisitation (navigating to a previously visited page) accounts for 58% and 61% of all page visits. Our study shows that page revisitation is now even more prevalent, accounting for 81% of page visits when calculated across all users.
This raises questions of how we can focus our sites, or individual pages, given how they are revisited, especially if a goal of the site is loyalty. Put another way, if users are loyal to certain pages, how should that affect the navigation?
Temporal aspects: The results show that browsing is rapidly interactive. Users often visit several pages within very short periods of time, implying that many (or most) pages are only displayed in the browser for a short period of time. Figure 3 shows that the most frequently occurring time gap between subsequent page visits was approximately one second, and that gaps of more than ten seconds were relatively rare.
Whereas Dillon discusses navigation as meaning in a context of info seeking, this result talks more about users having route knowledge, again with implications for navigation. A typical user comment: ‘I’ve never bookmarked the library’s search page. I keep forgetting because once I’m there I start my search rather than thinking to bookmark it. Anyway, I’ve got a good shortcut. First, I click `Home’ which takes me to the Department’s homepage, then I click on the link to the University’s homepage, and from there I click on `Departments’ and then `Libraries’. It takes quite a few clicks, but it doesn’t take too long.’
So when devising an interaction model, it’s good to consider the nature of the content and navigation as well as whether the users are new are repeat.
A community doesn’t exhibit homogeneous web use: These results show that there was a surprising lack of overlap in the pages visited by this fairly homogeneous community of users.
Conclusions: the authors offer now familiar recommendations, such as support revisitation, design pages to load quickly, shorten navigation paths, and minimize transient pages. Of course doing this is the real world is harder. Should all pages load quickly, or is it alright for ‘destination’ pages (with target content) to be larger? If we have a lot of information but must shorten navigation paths, should websites be smaller?
Thank you Andy and Bruce.
-
Tests Well With Others
In this brave new world where user-centered designers meet old-school info technologists, we can all live, love, and test together. Here’s how my usability testing jives with their testing, as I understand it:
What types of testing do we do?
- QA: technical check to make sure it works
- UAT (user acceptance testing): ensure the system meets the business requirements
- Usability: testing with end users
Who does it?
- QA: technical staff, but not the people who programmed it
- UAT: business staff. Originated in days when business people threw requirements over the wall to tech and needed to test what came back over the wall. In agency models it’s often rolled into design.
- Usability: dedicated usability peeps or the design staff, but preferably not the people who designed it
-
Design the Designless
OXO Good Grips are often cited as examples of insightful design. Having worked with someone who helped design them, it seems to me a case of someone actually designing, rather than just cobbling together the same old thing. OXO’s a great example to use when explaining the design process and its benefits because the improvement is so obvious. But the lack of any design thought in the previous product design is also obvious. I don’t mean to detract from the Good Grips design – it’s wonderful – but the traditional bent-metal thingies were practically designed to hurt the hand.
I’d like to think the impetus behind the design was much more sophisticated, that OXO also nailed the market and the timing, introducing better, more expensive kitchen tools just when the market was ready to spend on them, but apparently the origin was simply ergonomic; someone’s wife had arthritis.
-
Amazon Gold Box
There’s a classic Seinfeld episode where Kramer’s new phone number is one digit from Movie Fone’s, so, surrendering to the misdialed calls, he imitates the automated voice. When he can’t interpret the key presses he improvises, ‘Why don’t you just tell me the name of the movie you want to see?” Which is kinda what interfaces should let us do.
We have a running joke at work: when a navigation design gets away from what the user wants someone says, ‘Why don’t you just tell me what you want to see?‘
It seems like Amazon took this same approach… ‘Hey Victor, why don’t you just tell me which products you want in your Gold Box?’‘
-
IDEO’s Method Cards
I received my IDEO Method Cards, and they’re big, twice as big as your usual playing cards. The writing is good; short and sweet with funky photos on the reverse. The content isn’t earth shattering – each features a user research technique – but the format is quite handy and sure to stir up some new ideas at work.
A coup for IDEO is how – as in this Fast Company article – they position what could be considered an advertisement as a confident revelation of their methodology. Clever.
-
Extra-Company Resource Management
Walking down 87th St this morning I passed some construction workers hanging out on the sidewalk, as if they’re waiting for someone to pick them up for a job. This makes me think about new media workers…is there a reason we haven’t moved to a project-based model of employment like the movie industry uses, employing those who are just right for a particular product (does it suck?)? Would it be helpful to put a more flexible employment system in place?
Imagine combining P2P or local RDF with resource management apps. Workers indicate their availability simply with a tag on their site or on a central site and then this info is aggregated and syndicated to anyone who wants to see it. Perhaps companies combine it with their internal systems to think more flexibly about how to staff a project.
Lazy web, I summon thee!
-
AIfIA Tools
Back in the day we dreamed about exchanging deliverables so we could stop reinventing the wheel and start standing on each other’s shoulders. It’s odd something so basic as a repository hadn’t been established (I don’t think competitive advantage explains it in our open community). There is a collection of links on the IAwiki, and now AIfIA has launched a set of deliverable tools. It kicks off with a handful of top-notch documents from Erin Malone at AOL, sanitized and ready for reuse, which I think makes them a bit more useful than your average deliverable. I hope others will step up and put their stones in the soup.