Max Snyder from Electronic Ink, a design research and consulting firm in Philadelphia, came to speak to our Independent Study class about the methods and tools that are used to evaluate concepts. First, he outlined the two main criteria used to categorize the methods, they are: moderated vs. unmoderated, and remote vs. in-person. Examples included: surveys (unmoderated and remote), and eye tracking (moderated and in-person), and usability testing, which he referred to as a "catchall"– fitting into any of the categories.

Mr. Snyder loosely defined usability testing as showing an object to someone for their feedback–contrasting it with a focus group, which is about talking to people to get their feedback.

Concept Evaluation MID April 02 2013 - scrubbed_Page_06


Next, he talked about 5 things to consider when building a methodology for concept evaluation, they are:

  1. Business requirements
  2. Context of use
  3. Types of participants
  4. Objects for evaluation
  5. Team's appetite for methods (i.e. do they prefer more scientific methods? or more creative methods?)

Then, once the most appropriate method has been chosen and the methodology has been built, it is about determining the type of tools you want to use/have access to, to document and record the person testing the concept. Mr. Snyder shared a list of tools that he uses in his usability testing work at Electronic Ink, they are:

  • Voice recorder
  • Screen recorder
  • Video editing software
  • Digital camera
  • Remote viewing software (i.e. WebEx)
  • Web cam(s)
  • Protocol (discussion guide)
  • Mobile device testing rig
  • Eye-tracker

From the stories he shared throughout his talk, I extracted two key take-aways to keep in mind when doing my own concept evaluation. First, Mr. Snyder talked about including the client in the process of making the concept that gets tested with their customers, then having them observe the testing through a two-way mirror. This client involvement supports a greater understanding and adoption of the iteration phase inherent in the design process.

The second key take-away was about building rapport with the people that are coming in to evaluate the concepts. Mr. Snyder commented that the more at ease the person testing the concepts is, the more likely they are to be open and honest in their critique and feedback. He noted that one way that the person leading the usability testing session can build this rapport is through the opening "get to know you" type of questions. By remembering some of the personal details, like children's names, the person leading the session can later refer to those when they are trying to build a realistic, relatable scenario for the evaluator to imagine.

It was a great talk, and Mr. Snyder provided very useful information about concept evaluation, one of the most important steps in the design process.