Conducting the interviews
The LEAD Center interview protocols were created on the basis of specific research questions. These research questions were often designed so as to determine the degree to which the course professor's learning goals were achieved and to examine the nature of the students' learning process. Thus, if a potential user of these instruments has different student learning goals, some of the interview questions may be less appropriate than others.
All of the LEAD Center interview protocols included here are open-ended. In contrast to most surveys, open-ended interview protocols are designed to allow the respondent to speak freely about their experiences without being limited by predetermined response categories. Potential users of these instruments should keep in mind that the questions served as a guide to a "conversation" with the interview respondent; the questions were not asked verbatim or in the same order for each interview, as the interviewee was allowed to guide the conversation to some degree.
Interviews were conducted by LEAD Center researchers. In all cases these researchers were considered "third party" in that they were not members of the academic departments through which the course or program under evaluation was taught. Thus, the identity of student interviewees was not revealed to the course instructors, and students were informed that this was the case before agreeing to participate. This was important because it allowed the student interviewees to feel that their opinions would not affect their relations with their course instructors. In cases where no third party evaluator is available or desirable, we recommend that the interviews be conducted by someone who has no control over the students' grades. One possibility is for two professors to interview each others' students.
Analyzing the interviews
Sample results based on the use of LEAD Center interview protocols were obtained by analysis of verbatim transcripts of the audio-taped interviews. Although it is not necessary to transcribe the interviews, this does allow for more in-depth analysis of subtle themes. It is possible for the interviewer simply to take notes during the interviews and base their analysis on these notes.
The LEAD Center analysis process consists of developing inductive, thematic codes for the students' experiences. Generally, the researchers first read all of the transcripts. Then, they develop a rough coding scheme based on themes which appear in multiple interviews. Using this scheme, each individual transcript is coded. Then, interview excerpts are cut and pasted into an analysis document so that statements which are alike can be examined side by side. (Alternatively, a qualitative research software package may be used to sort and organize interview themes.) The coding scheme is gradually refined so that it represents a listing of the umbrella themes that capture the range of experiences discussed in the interviews. The analysis process is quantitative only in the sense that rough counts of the number of interviewees who spoke about a particular type of experience are developed. It differs from the traditional definition of quantitative analysis in that it does not tell one how many of the participants had a particular experience, but only how many chose to talk about it in the interview. Again, the interviewees are engaged in conversation, and therefore, not all issues are touched upon in each interview. Thus, though a small number of students may discuss an issue, the issue might be important in the experiences of a larger number or percentage of students.
Reporting Processes
The LEAD Center's reports include numerous interview excerpts so that the reader can hear the respondents voices when reading about the themes.
In order to protect respondents' identities, any identifying information (names, personal information, etc.) is removed.
Most of the themes from the student interviews were consistent with the survey data. However, in light of the small interview sample the qualitative findings should not be interpreted as representing the majority view, but simply as reflecting the views of the small interview sample. They are intended to raise further questions rather than to serve as "the final word" on the Topic-Oriented Approach Chem 104 student experience.
It is important to note that the student interviews were not used in isolation. They were used in concert with pre- and post- student surveys as well as classroom observations. This is not to say that interviews alone would not provide important information. It does suggest that in reading the sample results, one should keep in mind that we would place much less confidence in these findings had they not been supported by survey data from the majority of the class.
