To identify goals and task you could interview potential users or stakeholders. For your applications the scientists are appropriate stakeholders to interview. There are two types of interviewing process: structured and unstructured interviews. Structured interviews are preformed with a list of predefined questions and generally with expected answers. The structured interviews are easy to give, but they can only explore predefined subject matter. Unstructured interview take a lot of skill, but they can generate unanticipated goals or tasks. The interviewer needs to be well prepared for unstructured interviews. Semi-structured interviews combine features of both structured and unstructured interviews. The interviewer may have a list of prepared questions and ask spontaneous probing questions.
Questions
Question should be simple and not complex. Questions should invoke the right amount of elaboration. For example, a general or vague question may elicit either an answer that is too short or an answer that is too long. The language of your question should be understandable to the interviewee. Do not use technical jargon to ask your question. Do not go into elaborations to explain your jargon; instead ask the question in the language you know the interviewee can understand. Make you question neutral, do not express judgment in your question. Do not ask leading questions.
Open and Closed Questions: The questions in an interview can be open or closed. Closed question have expectation about the answer. For example question requiring only a yes or no answer is a closed question. Open questions do not have expectation on the type of answers. They are general are explorative questions and require the interviewee to give long answers. Generally, structured interviews use open questions and unstructured interviews use closed question.
Explorative and Probing Questions: Questions can have different goals. Question can be explorative, meaning the question seeks to find new material about the subject domain. Explorative questions are generally broad questions, such as “How do you use the smartphone to performing your tasks?” Probing questions, seek to elicit details about subject. For example after hearing the answer, “I take pictures and write notes using the smartphone” you might ask the probing question, “How do you take the picture?” or “Of what do you take pictures.”
Grounding Questions: I frequently use grounding questions. Grounding questions are probing questions that seek to find the facts behind an opinion or speculation. You will frequently find that when you are seeking facts that the answers you get are either an opinion or a speculation. For example asking, “Why are you holding the phone like that in your left hand?” You may get the answer, “Because it feels good,” which is an opinion. Generally we do not want our design based on the opinion from one user, so you would ask the grounding question, “Why does it feel good?” The answer, “The phone should be colored red” to the question, “What color should the phone be?” could be a speculation. To make certain the interviewer should ask the grounding question, “Why should the phone be colored red?” You may get answer like, “So that is easy to see the phone in the woods.” Now the designer knows the design goal should be to make the phone visible. Perhaps yellow is a better color. Or you could get the answer, “Uh, I like red.” Now interviewer knows the answer was an opinion and maybe the phone color should be the choice of the specific user. The main point is that we generally do not want our design based on the opinion of any individual user, so if you suspect that the answer is an opinion or speculation, gently ask a grounding question. You can learn two things from the answer of a grounding question, whether the answer was an opinion or speculation and if you are lucky the concrete fact behind the opinion or speculation.
There are other types of questions:
Scenario Question: Asking the interviewee to imagine a scenario that you describe then asking the interviewee what they would do.
Analytical Question: Asking the interviewee for a specific solution to a technical problem.
Process or Timeline Question: Asking the interview about steps in a process, or what happens next.
Detailing Question: Asking the interviewee for more details.
Abstraction/Generalization Question: Asking the interviewee to generalize on a detailed answer.
Interview Membership
Interviews can be with a single interviewee or with a group, called group interviews. To effectively interview a group the interviewer will want to get answers from all group members. This may require eliciting questions from shy members of the group or quieting talkative members group. This is a difficult skill. You need to make the shy individual secure, and you may need to be forceful with the talkative members. The interviewer can say up front, “I will want to hear answers from all of you, so from time to time I may ask questions directly of individuals.” When a talkative individual pauses during their answer, you can quickly ask a question directed to another member. Another solution to hear from all members is by conducting the interview as a round robin, asking to hear answers from all members to the same question.
The interviewer can be an individual or a group. Group interviewers require organization. A common organization of the group is by roles. One member of the interviewing group is the moderator, another questioner, and another recorder. The interviewing group can also be organized by domains, meaning each interview ask question in specific domains. For example, interviewing a doctor for a new medical records application, one interviewer will ask questions focused on the layout of the graphical interface and another interviewer will ask questions about how the records should be organized. The organization can be loose, meaning that all interviewers are asking questions in all domains. This organization is difficult to maintain because naturally one member of the interviewing group will assume the lead and ask all the questions. But the organization has the advantage that if a question is overlooked looked, the member of the group realizing that a question needs answering can immediately speak up to ask the question.
Preparing Interview
The interview will only be as successful as much as you prepare for the interview. If you conduct an interview without any preparation, you will discover that it dissolves into a casual conversation, and at its conclusion you really have not learned what you needed to learn for your design. Preparing for an interview is typing a document before the interview, and you walk into the interview with the typed document. Not only does the typed document serve the purpose of knowing what questions you will ask, but it will also tell you how you are progressing during the interview. I prefer using a typed document rather than hand written because during the interview I do not want to have to spend time or effort reading my notes. I want all my focus on the answers to the question and thinking about what question to ask next.
I call my preparation document the “pre-interview notes.” The first line of the document is the name of the interviewee and the general goal of the interview. The second line is the time and location of the interview. Following the top lines are any introductory statements I intend to make. For example it could be an explanation of the goal of the interview followed by a summary of the previous meeting or a status report. The rest of the document is a list of prepared questions. These questions could be in categories.
If it is a group that is interviewing, decide how you will organize your group. I recommend organizing by roles and have an agreement/understanding that any member may speak if the perceive that an important question has been missed.
Interview Procedure
Arriving: If your interview is at the interviewee’s office you should arrive at the exact specified time for the interview, but be prepared to wait for your interviewee to finish up their work or tidy their desk. If the interview is at a mutually agreed space arrived early and have you documents arranged. Dress so that you show respect for the interviewee. If dressed in shorts and flip-flops, the interviewee will assume that the interview is not important to you, so why should they bother with their answers. You do not need to overly dress. A key is to dress as well or a little bit better then you expect the interviewee will be dressed.
Introduction: Your interview should begin formally, so that the interview remains serious. If you begin with a joke than the entire interview will be a joke. Begin by giving your name and if appropriate your occupation. For example, “Hello, I am Joseph Smith, computer science student in the HCI course.” Allow the interviewee to give their name and say what they might need to say. After all the introductions, state the purpose of the interview.
Asking Questions: You should listen more than you talk. This is the best piece of advice I can give you besides preparing for the interview and the most important interviewing skill you can acquire. Remember your goal is to learn as much as you can from the interviewee, if the interviewee does not speak than how can you learn anything.
You should write notes during the interview. I find handwritten notes on the typed pre-interview document good. I do not suggesting typing because that becomes distracting. Do not be afraid to ask the interviewee to repeat a statement or answer while you deliberately write what they say. This gives the impression of value/importance to the interviewee.
But sometimes you will have to keep the interview on track. There are perhaps two reasons that you may have to keep the interview on track. Sometimes the interviewee will begin a long monolog about a tangential subject or elaborate on unimportant detail. Sometimes the interview will give quick answers without elaborating on important details. If the interviewee does not elaborate enough you can ask a grounding question or rephrase the question into a different type. For example, you can rephrase the question into a scenario to elucidate more questions. I frequently use this technique because frequently the details of the answer are obvious to interviewee but not to the interviewer and asking the interviewee to imagine the scenario causes them to ponder and remember more.
Controlling the interviewee during long monolog answers is more subtle because you do not want to insult the interviewee by interrupting and saying, “Those are unimportant details.” Probably the interviewee believes they are important details. It is important for the interviewer to recognize that the answer is going off track, but the interviewer should be patience. When the interviewee finishes with the monologue, be prepare and ask a question that brings the interview on track. There are some individual that once they begin talking it appears they will never end. In that case, when the interviewee pauses, you just ask your question. Generally they start another monologgue about the question you have just asked and will not notice what you have done.
Concluding: Generally interviews are for a fix amount of time. For the interviews you will conduct, you should summarize with the client what you have learned from the interview before the time is up. Summarizing can create an agreed understanding between the parties. Or summarizing allows the interviewee to correct any misunderstanding. After the summary, the interviewer can make arrangement for another interview or ask if it is possible to meet again. If appropriate this would be a good time to set a date or the process for making an appointment. Finally thank the interviewee. If a group is interviewing, every interviewer should quickly thank the interviewee.
Interview Document
The notes that you wrote during the interview should be typed up. There are two reasons. The typed document becomes readable and can be filed or posted on the web. Typically your notes are only phases and after a few days the phase will lose their meaning. More important, typing the document gives you time to think and remember more about the interview. You should convert the phases in your notes into complete sentences. The sentences give the phases context and consequently more meaning. You will discover that writing the sentence will help to recall more detail. Ultimately, you’ll only remember from the interview what you type up, so the more you type the more you will learn from the interview.
I name the interview document, “Post Interview Notes” and use the “Pre Interview Notes” as an outline for the interview document. The interview document should have the name of the interviewee and the interview time and location.
If there are successive meeting/interviews, you will be surprised how useful the interview document will be in preparing you for the next interview.
Cognitive Task Analysis (CTA)
The interview process given below is a simplified version of the incident-report cognitive task analysis (CTA), an unstructured interview process. It is modified incident-report CTA from “Working Minds” by Crandall, Klein and Hoffman. It was develop to elicit expert knowledge by recalling specific incidents. You will not ask the scientist to recall a specific incident; rather you will ask scientists to describe the general procedure for collecting field data. You cannot just ask the scientists to describe the procedures because they will forget details or will assume that you know the details. So, the interview goes through four sweeps of the incident (or in our case the procedure):
- Incident/Procedure identification -> generate general timeline of the procedure
- Timeline verification (or review procedure) -> refine timeline and find important events on the timeline
- Progressive deepening (or probing the procedure) -> learn more details of the procedure
- Could or should the app do this? -> get specifics design details
In general, interviewer asks the experts to explain the general procedure for a task. The interviewer lets the expert go through the procedure in generalities not asking for details. Instead the interviewer records the rough timeline on paper. You may have to keep the expert on track because frequently they wonder or become professorial. The interviewer can let the experts profess for a while, but soon bring them back on track by saying, “What do you next in the procedure?”
When the expert has finished with the procedure, the interviewer refers to their notes and reviews the rough timeline for the procedure. The expert will confirm, corrects or add more details to the timeline. During this portion of the interview the interviewer indicates important points on the timeline or adds to the timeline.
After reviewing the time line the interview asks probing questions about the important points on the time line. This gathers more information or details about the timeline or procedure.
Probing Questions
- Ask about before or after the important points
- Ask how, where, what or why for each important point. With experts (especially scientist) be careful about why questions because the question may cause them to profess.
- Ask for more details.
- Make suggestion. This assumes that you know some about the procedure, but after a few interviews or at the end of an extended interview you will have learned enough to make suggestions.
After asking probing questions, the interviewer asks “what if” questions, or in our case “can design the app like this?”
Example: Army App Simplified CTA Interview
Designers interview the commander.
Sweep 1: Generate timeline
Designer asks, “What is the procedure for a recon mission?”
Commander answers, “We go to the site and take photos and take notes.”
Designer records in notes:
Go to site
Take photos
Write Notes
Sweep 2: Review timeline and refine
Designer says, “You go to the site then you take a photo at the site. Also you write notes about the incident. Is that that it?”
Commander responds, “Yes that is basically correct. Oh, we’ll probably go to the next recon site.
Designer records in notes:
Go to recon site
Take photos of incident
Write Notes about incident
Go to next recon site
Sweep 3: Deepen the timeline with probing questions
Designer asks, “How you decide where to go?”
This probes before “Go to recon site.”
Scientist answers, “The day before I generate a list of sites from previous incident reports.”
Designer asks, “Is there something specific about recon site?”
This probes for details about “Go to recon site.”
Designer asks, “How do you get to site?”
This is a “how” question about “Go to recon site”
Commander answers, “We travel by foot using a map and compass.”
Design asks, “How many and what photos do you take at the site?”
This is a “what” and “how many” question about “Take photos of incident site.”
Commander answers, “We take many photo, general photo of the incident from different views and then photos of the surroundings.”
Design asks, “What do you record in your notes?”
This is a “what” question for “records notes.”
Commander answers, “We write the incident type, time, and location. Also, we describe the neighborhood.”
Designer asks, “What types of incidents are there?”
This ask more details about the “Record type of incidents”
Commander answers,
“Oh, there are disturbances, unrest, riot, terrorism, or then there are civil disturbances, disorderly conduct, thief, assault, and murder.”
At this point the interview the designer has recorded:
Prior generation of site list from previous incident reports
Go to recon site
Walk with map and compass
Take photos of incident
Multiple views of incident
Photos of the neighborhood
Write Notes about incident
Incident type:
disturbances, unrest, riot, terrorism, or then there are civil disturbances, disorderly conduct, thief, assault, and murder
Time
Location
Neighborhood
Go to next recon site
Sweep 4: How about questions:
Design asks, “Does the soldier need to choose the mission on the App?”
Commander, “Guess so, he might have to go on more than one mission.”
Design asks, “Should there be a navigation view on the app showing the direction and distance to the site?”
Commander, “Yes, that is a good idea. Maybe the app should have a map?”
Design asks, “Can the app have a drop down menu for the incident type? This saves the time to key.”
Commander, “Yes, but the soldier should be able to write notes.”
Design says, “The smart phone can record the time and gps location of photos.”
Commander, “That is good.”
The interview process not only deepens the timeline or the procedure but also generate design ideas for the app.
You should prepare for the interview. Guess at the timeline and important points on the timeline then generate some probing questions. Bring the list of questions. During the interviews all the group members will attend. So, two members can conduct the interview, taking turns, or injecting with questions. The rest of members can take notes that they share with the interviewer. Naturally, if note takers have questions, they should ask them.
Be prompt for the interview. Dress neatly. Be efficient with the time. Be courteous.