General information was gathered from an information form which new incoming students completed. All new students enrolling for the fall term of 1989 were included. Information on this form included such details as age, gender, average high school grades, estimated parental income, SAT and/or ACT scores, probable major, and sections dealing with 1) reasons for attending college, 2) hours per week spent in the last year on various activities and 3) goals and values. (See Appendix A.1 for a complete list of the questions from the form.)
In order to be able to match pre-test and post-test scores, means, etc., the information for this section was "cleaned-up" by matching surveys by social security number and including data only for those respondents who completed both surveys. After analyzing both the full pool of responses and the matched responses, it was determined that the second or partial set was close enough to the full pool to be representative. It is this second set of data that is reported in this document. This information was collected and analyzed in order to get an idea of the demographics of the class that the bibliographic instruction program evaluation would follow for four years. (See Appendix A.2 for detailed statistics.) An overall summary of that data follows.
The mean High School GPA as reported on the information form was between the grade of B and B+, with the most frequent response being an average grade of a B. Estimated parental income was between $15,000 and $25,000, with most students responding "from $10,000 to $15,000." Approximately 56% (108) of the respondents were female, 46% (85) male (total number = 193). Mean age was between 18 and 19, a fairly traditional sample with the most frequent answer being 18. Because there were many students who took only one of the standard entrance tests (SAT or ACT), there were many missing responses, which threw the statistics off considerably. Because of this, ACT and SAT scores were not analyzed. Probable college major also was not analyzed.
Two reasons for attending college elicited the highest responses, "to be able to get a better job" and "to learn more about things that interest me." These were followed closely by "to be able to make more money," "to gain a general education and appreciation of ideas" and "to prepare myself for graduate or professional school." Scoring lowest in this section were "there was nothing better to do," "I could not find a job" and "wanted to get away from home."
In the area of time spent on activities as a senior in high school, the response with the highest mean score was "socializing with friends," (6-10 hrs/week) followed by "studying/homework" (between 3-5 and 6-10 hrs/week). The least amount of time was spent on volunteer work (between 1 and 2 hrs/week) and partying (between 1 and 2 hrs/week).
Seven goals statements were most frequently responded to as being "very important," with no statement tipping the scale as being "essential." The highest mean was for the statement "helping others who are in difficulty." Some of the other high scores were for "being very well off financially," "raising a family," "helping to promote racial understanding," and "becoming an authority in my field."
While administered as a part of the student information form the statistics for the Hutchins Library Inventory (attitudinal and library skills questions) are discussed in the section of this document that covers the pre-test/post-test data.
Along with the standard data mentioned above, the Student Information Form included nine questions that reflected students' attitudes toward the library. At the same time that students completed the Student Information Form, a survey of basic library knowledge was administered. This survey (Hutchins Library Inventory), including the nine attitudinal questions was used as a pre-test (August 1989)/ post-test (November 1989). The post-test was administered in all sections of English 015, English 100 and Freshman Seminar, GST 105.
A summary of the responses and an analysis of the data follows. (See Appendix B.1 for copies of the pre-test/post-test.) Question numbers refer to the numbers assigned to the questions for coding purposes (added to the left of each question on the copies), not the numbers printed on the student copy of the inventory, since the printed numbers changed from pre-test to post-test.
PRE-TEST POST-TEST Section I (1 = yes, 0 = no) Mean Mean Q.1 (instruction) .861 .932 Q.2 (public) .295 .233 Q.3 (high school) .731 .855 Q.4 (Berea) .171 .573 Q.5 (other college, etc.) .114 .125 Section II (1 = strongly agree...5 = strongly disagree) (See App. B.2 - see note below) Q.5 a (pre-test 41) I feel comfortable... 1.746 2.109 Q.5 b (pre-test 42) I feel confident... 2.119 2.422 Q.5 c (pre-test 43) I expect to research... 1.306 1.477 d (pre-test 44) I expect to study... 1.845 2.652 f (pre-test 46) I expect ...recreational read.... 2.834 3.071 Q.5 g (pre-test 47) Librarians are good source... 1.518 1.844 h (pre-test 48) Librarians are approachable... 1.979 2.203 Q.5 i (pre-test 49) Information skills important... 1.275 1.622
PRE-TEST POST-TEST question % no. total# % no. total# net % correct correct response correct correct response change Section III A 6 29.3 55 188 36.3 69 190 7.0 7 10.6 20 189 23.2 44 190 12.6 8 4.7 9 190 47.6 90 189 42.9 9 16.8 32 190 26.3 50 190 9.5 10 70.4 133 189 74.2 141 190 3.8 11 88.4 168 190 89.9 170 189 1.5 12 46.0 86 187 73.5 139 189 27.5 13 8.6 16 187 21.8 41 188 13.2 14 30.7 58 189 43.4 82 189 12.7 15 29.6 56 189 24.9 47 189 -4.7 Section III B 16 90.6 174 192 88.0 169 192 -2.6 17 93.8 180 192 90.1 173 192 -3.7 18 43.2 83 192 53.1 102 192 9.9 19 78.5 150 191 82.2 157 191 3.7 20 80.2 154 192 88.0 168 191 7.8 21 45.5 86 189 61.3 117 191 15.8 22 30.5 58 190 37.7 72 191 7.2 23 41.4 79 191 44.0 84 191 2.6 24 15.9 30 189 23.2 44 190 7.3 25 33.2 63 190 43.7 83 190 10.5 Section III C (See Appendix B.3) 26 84.7 149 176 88.8 167 188 4.1 27 77.7 146 188 71.6 136 190 -6.1 28 69.6 133 191 77.5 148 191 7.9
PRE-TEST POST-TEST question % no. total# % no. total# net % correct correct response correct correct response change 8 4.7 9 190 47.6 90 189 42.90 12 46.0 86 187 73.5 139 189 27.50 21 45.5 86 189 61.3 117 191 15.80 13 8.6 16 187 21.8 41 188 13.20 14 30.7 58 189 43.4 82 189 12.70 7 10.6 20 189 23.2 44 190 12.60 25 33.2 63 190 43.7 83 190 10.50 18 43.2 83 192 53.1 102 192 9.90 9 16.8 32 190 26.3 50 190 9.50 28 69.6 133 191 77.5 148 191 7.90 20 80.2 154 192 88.0 168 191 7.80 24 15.9 30 189 23.2 44 190 7.30 22 30.5 58 190 37.7 72 191 7.20 6 29.3 55 188 36.3 69 190 7.00 26 84.7 149 176 88.8 167 188 4.10 10 70.4 133 189 74.2 141 190 3.80 19 78.5 150 191 82.2 157 191 3.70 23 41.4 79 191 44.0 84 191 2.60 11 88.4 168 190 89.9 170 189 1.50 16 90.6 174 192 88.0 169 192 -2.60 17 93.8 180 192 90.1 173 192 -3.70 15 29.6 56 189 24.9 47 189 -4.70 27 77.7 146 188 71.6 136 190 -6.10
SURVEY 1 SURVEY 2 (1 = answer correct, 0 = answer incorrect) Mean Mean Sum 1 - library knowledge, sect. I (Q6 : Q15) 3.332 4.595 Sum 2 - library knowledge, sect. II (Q16 : Q25) 5.505 6.089 Sum 3 - library knowledge, sect. III (Q26 : Q28) 2.241 2.361 Sum 4 - all three sections (Q6 : Q28) 11.031 12.917 Note: Perfect score = 23 (See Appendix B.4 for more detail. This Appendix is not available as a WWW document - please contact the authors for copies.)
The first section of the survey (Q1:Q5) was an informational question to determine how much prior formal library instruction students had received and in what setting. Differences in responses to Q1, Q2, Q3 and Q5 are not particularly significant and probably can be explained by a variance in the student's memory. However, the change in the response to Q4 (instruction at Berea College) is a direct reflection of the library's instruction program. Some of the students were enrolled in ENG 015 and ENG 100, which do not necessarily have library instruction included, while others were in GST 105 (Freshman Seminar) which had two formal library instruction sessions scheduled as a part of the course. More than half of the students had received instruction by the end of the fall semester, a reflection of the percentage of students enrolled in the various courses used to administer the survey (ENG 015, ENG 100 and GST 105).
In the second section, students were asked to respond to nine attitudinal statements about the library. (Q5a:Q5i - see Appendix B.2 for further detail. This Appendix is not available as a WWW document - please contact the authors for copies.) On the pre-test, most students seemed to feel fairly comfortable and confident in their library interactions, agreeing that they planned to do research and studying in the library, along with a little bit of recreational reading. They did not see socializing as a library activity.
In the post-test, the mean score for the statements regarding comfort and confidence increased, signifying less agreement with the two statements. Because most of Berea's students have had little exposure to a large library, even one the size of Hutchins, they may have found themselves a little overwhelmed. The size of the building and the collection, as well as the requirement to accomplish complex tasks with information sources (especially in Freshman Seminar), might all contribute to a student feeling less confident and comfortable in the library after the first semester.
The statements reflecting anticipated types of use of the library did not change very significantly from pre-test to post-test. The largest change occurred with the statement about studying in the library. Students unfamiliar with Hutchins may have thought it would be like monitored study halls in high school - quiet and a good place to study. However, once they became familiar with Berea's library, they found that the building (as it then existed) had some significant noise problems, making it more difficult to concentrate and study than they had initially supposed. When asked at the end of the semester about anticipated study in the library, the change in students' responses may reflect experiences with noise problems during the semester. The 1991 addition and renovation have helped to solve the noise level problem and to create an atmosphere more conducive to study.
The last two attitudinal statements reflected a student's perception of librarians. The scores on these two statements did increase, however the majority of answers were still "agree" or "strongly agree."
There were three parts to this section: 1) ten questions matching types of information with sections or major library finding aids - serial record, card catalog, etc. 2) ten questions matching specific types of information with specific types of references/library resources - almanac, bibliography, dictionary, etc. and 3) three completion questions - name a periodical, recognize use for geographic source (atlas, globe or map), and recognize definition of an autobiography. (See Appendix B.3 for detailed results from completion questions.)
The overall mean score for the first part of section three increased from 3.33 to 4.595 (highest possible score = 10). The question that had the best gain from pre-test to post-test was question 8, which dealt with the serial record, a term students might not have known in high school and one that might vary from library to library. The percentage of correct answers on the pre-test was 4.7, while the corresponding figure for the post-test was 47.6. This question had the lowest correct response rate, not only for part one, but also for part two on the pre-test and moved to the third highest rate on the part one questions and eighth out of 20 overall on the post-test. Experience in using the library over the semester and instruction in use of the library were important in increasing the correct response rate for this question. One question from part one did show a decrease in percentage points; the question dealing with location of the text of Kentucky laws (Q.15) decreased from 29.6 to 24.9 percent correct.
On the second part of section three, the mean score increased from 5.5 to 6.089 (highest possible score = 10). Interestingly, the questions in this part retained their ranking in percentage correct scores from pre-test to post-test, while showing overall an increase in scores. Three of the questions from part two did loose points from the pre-test to the post-test, but they all had scores of above 70% correct initially and showed, at most, a decrease of 6.1 percentage points.
In comparing part one (sections of the library and library finding aids) and part two (specific types of references/library resources), the questions from part one showed the greatest increase. Since these questions dealt with terminology and sections of a library (Hutchins in particular), they could be seen as being more general and therefore, perhaps, more easily improved through introduction to and experience with a particular library. Questions from part two dealt with specific types of information and the types of sources needed to answer the need. While the mean score for the second part was higher than that for part one on both the pre-test and the post-test, the overall improvement was not as great for part two. Of the seven questions from parts one and two that had an increase of 10 percentage points or more, five were from part one and two were from part two.
The highest percentage correct for any particular question was 93.8 on the pre-test. This question asked the student to identify the card catalog as the most appropriate source to consult to find a particular book.
In part three, which consisted of the completion questions, students had the highest success rate. With a possible high of three (3), the mean for the pre-test was 2.24 and 2.36 for the post-test. Sixty-nine (69) percent or above answered correctly for each of these three questions.
In looking at the specific responses to these three questions, the composition of the students in Freshman Seminar classes must be taken into account. Because Freshman Seminar generally includes some sophomores, the specific responses on this section include those of students who did not take the pre-test. The analysis given above for the first two parts of section three could be made based upon scores for identical groups on the pre-test and post-test. The specific answers for the last three questions (part three of section three) were not indicated in the database, only whether correct or not. Specific answers were gathered from all returned surveys (actual paper copy) and were not identified with a particular respondent. Additional work remains to be done in order to identify only those answers identified as belonging to respondents of both pre-test and post-test.
Keeping that fact in mind, what follows is, at best, a tentative and very general analysis. The answers given for Q.26 (title of a periodical) showed greater diversity in the post-test. Seventy-four (74) distinct answers were included on the pre-test, while 99 were listed on the post-test. The type of periodical listed also showed a change from pre-test to post-test. Time, Newsweek, Sports Illustrated, and U.S. News and World Report were still among the top five answers, but 72 titles were included on the post-test that were not on the pre-test. This shows an awareness of an increasingly diverse and more scholarly level of periodical. (Titles unique to the post-test include Studies in Short Fiction, Business Week, The Economist, Hastings Center Report, Journal of Housing, etc.)
A more indepth analysis of the last two questions on the survey will not be made until a distinction can be made between those responses given by the group identified as having completed both the pre-test and the post-test and those returning only one or the other. For detailed information on part three, see Appendix B.3.
Also examined in this section of the evaluation was the possible correlation between pre-test and/or post-test score and a variety of factors, as self-reported on the student information form. Of the various factors examined, only that of high school grade point average seemed to have even a slight correlation with overall score on either pre-test (.295, covariance of 1.535) or post-test (.333, covariance of 1.988). The following figures are a result of testing for correlation between scores on the various sections, as represented by Sum. 1, Sum. 2, etc. (See Appendix B.1 for Questions included in each Sum.)
PRE-TEST Sum. 1 Sum. 2 Sum. 3 Sum.4 (total) Sum. 1 .319 .329 .774 Sum. 2 .246 .799 Sum. 3 .571 POST-TEST Sum. 1 Sum. 2 Sum. 3 Sum.4 (total) Sum. 1 .378 .224 .824 Sum. 2 .329 .801 Sum. 3 .503 ONE-TAILED T-TEST t-value sign. level pre-test post-test (score) mean mean Sum. 1 -8.844 .0005 3.33 4.595 Sum. 2 -4.209 .0005 5.500 6.089 Sum. 3 -1.995 .01<>.025 2.24 2.36 Sum. 4 -8.188 .0005 11.03 12.91 (Total) df=192 critical value (alpha=.0005) = 3.291
Detailed and indepth analysis of the data gathered through this process will continue in order to gain more fully from the evaluation process. Replication of this study will also be considered in order to assess the impact of changes in the B.I. program.
Susan_Henthorn@berea.edu
mroyse@utk.edu