Writer's Workshop Critique Format

Looking for a Lighweight Way to Receive Peer Feedback

In the past I attempted to improve the quality of information architecture work on the department level using heuristic analysis. It's relatively easy to do, but in reality I found analysis sessions difficult to run because 1) the presentation and feedback process is time consuming due to the scope problem: IA covers a large problem area, and 2) heuristics alone are unstructured, possibly causing blank-sheet-of-paper syndrome.

I tried accomplishing similar results using a format borrowed from writing groups, described below. We used it at the design patterns workshop at CHI 2000 and it was very successful.

To address the scope problem, we'd only review an approach instead of an entire architecture. An approach could be summarized by touching on

...and be expressed in under 20 minutes (figuring the entire exercise could run about an hour). The idea is that suggestions on an approach should help guide the IA through the rest of the architecture.

Writer's Workshop Format

This version contains my modifications to include heuristics

In preparation, all critics have familiarized themselves with the IA approach before the workshop. One of the critics, the facilitator, provides an initial welcoming, the author first reads a part of her work to the authors, to remind everybody there's a person with feelings behind the work. After this introduction, the author fades into the background and attends the following discussion without interfering (called a fly on the wall). One of the critics now summarizes the paper in his own words and the others can add to this summary. Next, the critics refer to the heuristics and offer positive comments on the submission, and subsequently constructive suggestions (what could be improved) are collected. The discussion ends with a summary of the good points of the paper (this sandwich technique avoids a negative lasting impression). After this, the author is welcomed back into the group and allowed to ask questions if some comments were not clear to her, or if she wishes to see another aspect of her paper discussed. She is not allowed, however, to defend her work at this time. As this whole discussion can be a bit harsh at times, the author is finally applauded for her work (and courage to submit it), and, to round out the activity with levity, somebody closes the session with an entertaining unrelated story.

In summary, the steps are:

  • Preparation
  • Welcoming
  • Author reads, then becomes a fly on the wall
  • Summary
  • Positive comments
  • Constructive comments
  • Summary of good points
  • Author's questions
  • Applause
  • Unrelated story

I've tried it a couple times with limited success. I'm interested to know if others have used it or if it could work in an online critique format.

Thanks to Jan Borchers for sharing this format with me.

Thursday, April 4, 2002 | Permalink | Filed in Process