In the fall of 1991, work began on the process of conducting focus group interviews with a sample of students from the class of 1993. The sample was drawn from the total number of students who had completed both pre-test and post-test in a usable manner and were still enrolled in school. Students who had worked in the library were discarded from the sample pool because of potential for bias. Instead, eight student library workers from this class were selected, trained and used as student interviewers in the focus group sessions.
The goals and objectives for the bibliographic instruction program were used as touchstones for establishing the goals of this segment of the evaluation project. (See Purpose, Goals & Objectives, Criteria for a copy of goals and objectives.) The major purpose was to discover what students thought had been gained from their library experiences. Instead of matching specific questions and specific goals, the focus was directed toward developing questions that would elicit the most candid, informative and helpful comments. This section of the larger evaluation process would intentionally be more qualitative and exploratory in nature than some of the other components (new student pre-test/post-test, Senior Requirement paper bibliography assessment, etc.).
The goal was to have fifteen groups, with four students interviewed in each group; the sample size was set at 60 accordingly. In order to be representative of the students' experiences, the sample was drawn evenly from the three English or Seminar entry courses the students could have had as freshman (ENG 015, ENG 100, or GST 105).
From an initial list of questions, the script for the sessions was drafted during fall of 1991. The focus of the sessions was to be three-fold: 1) to discover what the students remembered about the library instruction they had experienced, 2) to informally assess their knowledge of library research and 3) to discover what the students thought the role of the library had been in their education as well as its future role.
The initial draft was used in a mock session with four of the eight library workers as interviewees and the author as interviewer. Responses from the student participants lead to some modifications in the script. In order to get comments from a student on the actual process of interviewing, the process was repeated, this time training and using one of the four students from the first mock session to conduct a second mock session with the remaining four library workers. After listening to tapes of the sessions and comments of the students involved, the script was revised and printed in its final form (See Appendix D.1).
Also during the fall, schedules were obtained from the registrar for the sample student population to facilitate scheduling of interview sessions. Also obtained from the registrar for each individual was a listing of courses taken to that point. The list was to serve as a reminder to the student of the courses taken and to refresh the student's memory of which courses might have included library instruction. In early January, this list was sent with an initial letter to the sample population informing them of the process and asking for their participation in a particular session. (See Appendix D.2. This Appendix is not available as a WWW document - please contact the authors for copies.) The second week, a reminder letter was sent to the sample students. If no reply had been sent by the third week, phone calls were made to try to encourage participation.
A training session for the student interviewers was held during the second week of January, with the sessions held the third and fourth week of January. (See Appendix D.3 for training documents. This Appendix is not available as a WWW document - please contact the authors for copies.) Ideally, each session was to include four student informants. If only one student appeared for the session, the student informant was given a choice of whether or not s/he would continue with the session as scheduled. This was done so that if a student felt comfortable enough to respond to the questions as the only informant in the session, they could continue; but if the student would feel uncomfortable in that setting, s/he could choose to try to re-schedule and participate in another session. The response rate was disappointing (36.7%); however, many helpful comments were made.
The long process of tape transcription was begun in the spring of 1992. In the fall of 1992, Institutional Testing was consulted regarding the summary process to be used with the tape transcripts, and that process of summary and analysis was started.
By spring of 1993, the summary process was complete. The sessions were summarized both question by question (See Appendix D.4) and student by student (See Appendix D.5. This Appendix is not available as a WWW document - please contact the authors for copies.) The instruction staff met to review the summary comments in order to facilitate the analysis of the data. (See Appendix D.7 for timeline for focus group interview process. This Appendix is not available as a WWW document - please contact the authors for copies.)
From the comments generated by the interview process, the majority of the students seemed comfortable in making candid remarks. For some comments, more information might have been gained by the interviewer using follow-up questions to probe more specifically, but because of the relative inexperience of the interviewers this did not happen as often as desired. Although additional indepth comments would have been helpful, information gathered through this process proved extremely helpful in discovering students' views of the bibliographic instruction program. The results can be categorized in the following manner: 1) what was learned, 2) what changes have resulted from the information gathered and 3) what suggestions and/or challenges remain for the future.
Some of what was learned through this process was new to the library staff, while other information differed very little from what had been assumed through informal contact with various students. Some of the comments were positive, showing that goals were being met and successful strategies are already in place. There were a few mild surprises in the comments also, sometimes pointing to differences between the librarians' and students' perceptions of library sessions while at other times revealing weaknesses in the overall process.
On the positive side, students mentioned Special Collections as holding interesting material that was useful for specific types of research. Also, students overall found the library staff to be helpful and good sources when one is stumped or not sure what to do next in the research process, although one or two also mentioned interactions with 'grouchy' library staff.
When a student had had at least one successful library experience, comments seemed to indicate a higher comfort level and appreciation for the rewards of library research. However, the freshman seminar experience with the research paper was often viewed as anything but a successful experience. Most often mentioned were words such as 'boring,' 'dull,' and 'repetitive.' Along with the sessions being perceived as boring, the worksheets that students were required to complete were frequently viewed simply as a disconnected exercise to complete, and one that caused very high stress levels. To add yet a third negative experience, a very common answer to the question asking for 'the most frustrating library experience' was the problem of using indexes to find periodical articles and then finding that, for one reason or another, the article was not in the library (no subscription to periodical or misshelved/missing volumes). The general education curriculum reform will provide a good opportunity to avoid some of the more obvious problems inherent in the assignment of a major research paper in a freshman level course.
The repetitive nature of library instruction was probably mentioned because of two factors: 1) students felt that they already knew how to use the library - this was all a repeat of things learned in high school, and 2) because of the nature of Freshman Composition, Freshman Seminar and the later, upper level courses, students often heard information in a similar format, i.e., classroom lecture in one of the library's classrooms.
The first factor is hard to avoid. Many of Berea's students come to college with very little library experience, especially with a library larger than perhaps a large sized classroom. To assure that all students are exposed to the basic principles in library use, it has been seen as critical that all students be given an opportunity for library instruction, fairly early in their college career. For some, there will be a certain amount of repetition, however, the key is to bring all students to a certain level of acquaintance with library research and then draw them further into the process. Most students will be asked to complete much more rigorous library research in their college courses than had been expected of them in high school, and this demands more advanced research skills.
The feeling of repetition based upon format (although content did vary) is somewhat easier to address. Some changes have already been made as a result of this type of comment. When instructors requested library sessions for Freshman Composition classes, an effort was made to deal with the class in the reference section exclusively, and not present something that would bear even a slight resemblance to a lecture format. Usually at least two reference librarians would each interact with smaller sections of the class, providing a less formal setting and a better librarian to student ratio. Also incorporated in this approach was the suggestion from several students that a more hands-on approach be used in library instruction.
One of the questions sought information about the students' idea of the library's role in their education, specifically, in their ability to make sound judgments and in their development of attitudes. For several students, the phrase 'humbly Christian,' contained in the question on the relationship of the library to overall college goals, appeared to be a bit of a red flag, eliciting strong negative comments. For others, there seemed to be a good understanding of the connection between the understanding of global goals of the institution and something as specific as learning how to use the library. Most students did seem to have gained some sense of the value of knowledge, intellectual abilities, and skills needed to use information. They did articulate a value for the library beyond supporting individual course assignments.
The question dealing with a group process was included in the hope of discovering students' ability to articulate procedures involved in conducting library research. The comments revealed that most students had little sense of what to do when they didn't find any information; many made the immediate jump to the assumption that the library didn't contain the needed information. The research process, as revealed by responses to this question, was very superficial; steps in the process were not given in any detail. This could be a result of the inexperience of the interviewers in drawing out further detail from informants or it could be a result of the inability of the informants to clearly articulate and organize the steps involved in the library research process.
An overall positive impression from several of the students' was that library research is work, but it's rather fun to look for, to dig for, and to find needed information. Again, the research process was not described or detailed in any depth, but was mentioned as one of their primary uses of the library.
Students need to see the relevance of library instruction and research, regardless of the level of sophistication of their library skills. One of the primary ways this can be achieved is through careful consultation and interaction between the teaching faculty and the instruction librarians. The timing of library instruction sessions and the direct application of skills to assignments are critical in order for students to discover optimum value in the library research process.
If students can grasp the individual pieces of the research process, they will be more likely to understand that the research process does not take on one consistent pattern, but instead is shaped by the information needs of each individual research task. Goal statements should reflect the need to master both use of individual types of sources and the organization of the information research process.
Finally, more opportunities for students to develop a feeling of success in their early library experiences should be implemented. The two most frequent comments elicited from these student interviews related to 1) repetitive, boring sessions and 2) the feeling of being overwhelmed by the whole process. Neither situation creates an opportunity for students to feel successful in their initial college library experiences. Through the development of the new general education courses, the librarians and teaching faculty have a prime opportunity to create progressive, organic library assignments in such a way as to build students' library skills and experiences so that they will have a greater feeling of success, especially in their initial encounters. It's hard to improve upon the feeling of success as a motivation tool in approaching future assignments.
Susan_Henthorn@berea.edu
mroyse@utk.edu