Diff for "DiscussionGuide"

Differences between revisions 5 and 7 (spanning 2 versions)

This page contains guidelines for conducting user interviews and a Discussion Guide which outlines topics of interest that we would like to cover in the interviews. The Discussion Guide should be used as an informal list of topics we would like to cover in interviews. The guidelines below are paraphrased from planning documents of user studies we conducted at HP Labs and later labs at Agilent Labs. As guidelines, they are meant to provide an outline of some basic principles and techniques we found useful, not a formal set of rules to be followed.

Suggested Guidelines for Interviewing

We found it most productive to interview in teams, or at least in pairs. In teams, one person can be responsible for note-taking while the other(s) can devote their full attention to the conversation. Another advantage of team interviewing is that the sharing of reactions in debriefing can help synthesize conclusions and new ideas.

Before each interview, we found it a good idea to review among ourselves what the goals of the interview are and recall what questions or issues we especially wanted to cover in the particular meeting. The topics in the Discussion Guide can serve as an ongoing list of these important issues. They are best used as aids for moving the conversation along, rather than as a questionaire. It should not be essential to cover all questions in the guidelines in all interviews; rather it is more productive to let the interview take its course and to capture the information in the users' own words. Interviews should be informal and open-ended. We will have specific ideas in mind of what we want to cover, but it is essential to allow these conversations to unfold in their own way, rather than presenting a laundry list of questions. In fact, it is a good idea to put aside any papers with written questions or topics before the interview starts (after reviewing them), since that will help us listen more closely to the cues of the people we are talking to. We need to follow those cues if we want to understand what ís important to the people we interview.

We found it helpful to listen especially carefully to the language people use, and to try to adopt their terminology in interviews and other conversations whenever we can. Noticing the differences between the terminology of people we interview and the terminology of our own group of developers (or what we understand to be conventional bioinformatics language) can be a fruitful source of insight about the way users go about doing things.

We found it useful to debrief among ourselves right after the interview. In the debrief, we can call attention to any interesting things that surfaced, see if any questions have arisen for follow-up interviews or observations, and discuss the logistics of the interview to see if anything should be changed for next time.

Some user research practitioners tape all interviews. We did this at HP Labs for several years, but eventually abandoned the practice favor of concentrating on taking good notes and writing them up as soon as possible after the interviews. We found that this was just as effective as taping and transcribing interviews, while being lower overhead and avoiding the problems of people feeling self-conscious when being taped.

It's essential to take good notes during an interview and, even more important, to write up these notes as soon as reasonable after the interview, while the conversation is still fresh in our minds. Such 'field notes' are very useful for setting the scene, summarizing the important features of a conversation, providing some material for reflection, and pointing to next steps.

Discussion Guide

Introduce ourselves
What kind of work do you do and how do you use Cytoscape?
- What parts of the system do you use most?
  - Do you spend most of your time :
    - editing networks,
    - setting visual attributes,
    - editing attribute data,
    - working with plugins, etc
- How do you run (start) cytoscape
  - running a command line script,
  - double-clicking an icon, menu item).
  - Is this easy for you?
  - Is this how you run other programs you use on a regular basis?
Installation experience.
User interface and usability. Are the GUI features intuitive?
- Do the menus make sense?
- Do you like cytopanels (i.e. frames within frames, or separate windows)?
- Is the relationship of the graph model to attribute data to visualization clear?
Is the mapping of network data to visual attributes easy?
Performance & responsiveness: Operations on large networks too slow?
Features: which are useful? useless?
- Are there any features that you do NOT use? If so why?
- Are there any features that you find confusing or particularly annoying?
Plugins: Are they easy to use?
- Do plugins work for you? Are they easy to use and integrate with cytoscape?
Can you suggest other people that we might want to interview?

-  ← Revision 5 as of 2006-02-16 06:49:30 →
  Size: 5428
  Editor: cosiapat1
  Comment:
+  ← Revision 7 as of 2006-02-16 07:06:59 →
  Size: 5134
  Editor: cosiapat1
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-This page contains guidelines for conducting user interviews and a '''Discussion Guide''' which outlines topics of interest that we would like to cover in the interviews.  The Discussion Guide should be used as an informal list of topics we would like to cover in interviews.   The guidelines below are paraphrased from planning documents user studies we conducted at HP Labs and later labs at Agilent Labs.  As guidelines, they are meant to outline some basic principles and techniques we found useful, not a formal set of rules to be followed.
+This page contains guidelines for conducting user interviews and a '''Discussion Guide''' which outlines topics of interest that we would like to cover in the interviews.  The Discussion Guide should be used as an informal list of topics we would like to cover in interviews.   The guidelines below are paraphrased from planning documents of user studies we conducted at HP Labs and later labs at Agilent Labs.  As guidelines, they are meant to provide an outline of some basic principles and techniques we found useful, not a formal set of rules to be followed.
 Line 3:
-== Guidelines ==
+== Suggested Guidelines for Interviewing ==
 Line 5:
-We found it best to interview in teams, or at least in pairs.  In teams, one person can be responsible for note-taking while the other(s) can devote their full attention to the conversation.
+We found it most productive to interview in teams, or at least in pairs.  In teams, one person can be responsible for note-taking while the other(s) can devote their full attention to the conversation.  Another advantage of team interviewing is that the sharing of reactions in debriefing can help synthesize conclusions and new ideas.
 Line 7:
-Before each interview, it is a good idea to review among ourselves what the goals of the interview are and recall what questions or issues we especially want to cover in this particular meeting.  The topics in the Discussion Guide can serve as an ongoing list of these important issues.   They are best used as a means of stimulating the conversation, rather than as a questionaire.  It should not be essential to cover all questions in the guidelines in all interviews.  Then we let the interview take its course.  Our interviews are meant to be informal and open-ended.  We will have specific ideas in mind of what we want to cover, but it is essential to allow these conversations to unfold in their own way, rather than presenting a laundry list of questions.  In fact, itís a good idea to put aside any papers with written questions or topics before the interview starts (after reviewing them), since that will help us listen more closely to the cues of the people weíre talking to.  We need to follow those cues if we want to understand whatís important to the people in our partner group.
+Before each interview, we found it a good idea to review among ourselves what the goals of the interview are and recall what questions or issues we especially wanted to cover in the particular meeting.  The topics in the Discussion Guide can serve as an ongoing list of these important issues.   They are best used as aids for moving the conversation along, rather than as a questionaire.  It should not be essential to cover all questions in the guidelines in all interviews; rather it is more productive to let the interview take its course and to capture the information in the users' own words.  Interviews should be informal and open-ended.  We will have specific ideas in mind of what we want to cover, but it is essential to allow these conversations to unfold in their own way, rather than presenting a laundry list of questions.  In fact, it is a good idea to put aside any papers with written questions or topics before the interview starts (after reviewing them), since that will help us listen more closely to the cues of the people we are talking to.  We need to follow those cues if we want to understand what ís important to the people we interview.
 Line 9:
-Weíll need to listen especially carefully to the language people use, and we should try to adopt their terminology in interviews and other conversations whenever we can.  Noticing the differences between the local terminology of our partner group and the terminology of our own group (or what we understand to be industry-wide language) can be a fruitful source of insight about local ways of doing things.
+We found it helpful to listen especially carefully to the language people use, and to try to adopt their terminology in interviews and other conversations whenever we can.  Noticing the differences between the terminology of people we interview and the terminology of our own group of developers (or what we understand to be conventional bioinformatics language) can be a fruitful source of insight about the way users go about doing things.
 Line 11:
-When we are doing a team interview, itís useful to debrief among ourselves right after the interview.  In the debrief, we can call attention to any interesting things that surfaced, see if any questions have arisen for follow-up interviews or observations, and discuss the logistics of the interview to see if anything should be changed for next time.
+We found it useful to debrief among ourselves right after the interview.  In the debrief, we can call attention to any interesting things that surfaced, see if any questions have arisen for follow-up interviews or observations, and discuss the logistics of the interview to see if anything should be changed for next time.
 Line 13:
-We will tape all interviews.  (This is something weíll discuss with our partner group at the beginning of the study.)  Each of us will have our own tape recorder, and weíll keep a common supply of tapes and batteries to draw from.  The person who is being interviewed should be reminded at the beginning of the interview that they can ask for the tape recorder to be turned off at any time.  (Sometimes people ask for the tape to be turned off when they want to say something confidential, and then theyíre willing for it to be turned on again.)  It is a good practice to put in fresh batteries well before the battery indicator says the power is low.  (Itís awful to have a great interview and find out later that your tape recorder wasnít working.)
+Some user research practitioners tape all interviews.  We did this at HP Labs for several years, but eventually abandoned the practice favor of concentrating on taking good notes and writing them up as soon as possible after the interviews.  We found that this was just as effective as taping and transcribing interviews, while being lower overhead and avoiding the problems of people feeling self-conscious when being taped.
 Line 15:
-Itís a good idea to take notes during interviews, even when the tape recorder is running.  This is helpful for two reasons.  One reason is that if the recording doesnít work, the tape is lost, or there is some other disaster, we have some record of the conversation.  The other reason is that the notes give us some material from which to produce a fieldnote right away, before the tape is transcribed.  Even when we have a transcript, a fieldnote is useful for setting the scene, summarizing the important features of a conversation, providing some material for reflection, and pointing to next steps.
+It's essential to take good notes during an interview and, even more important, to write up these notes as soon as reasonable after the interview, while the conversation is still fresh in our minds.   Such 'field notes' are very useful for setting the scene, summarizing the important features of a conversation, providing some material for reflection, and pointing to next steps.