Student Evaluations: Widespread And Controversial

Jerome Hart Photography CONTROVERSIAL: Widespread use of student evaluations on campuses has become a hot topic, according to Terry Favero, an assistant professor of biology who teaches human anatomy at the University of Portland in Oregon. Teaching evaluations have become a rite of passage in nearly all college courses. Shortly before the final exam, students typically complete a standardized form that asks general questions on aspects such as course organization, level of difficulty, quality of lectures, grading fairness, and instructor accessibility; in many cases, students can add specific comments. "The idea of student evaluations as an assessment tool to help instructors figure out how well they are teaching has become a hot topic on most campuses," relates Terry Favero, an assistant professor of biology who teaches human anatomy at the University of Portland in Oregon. Fueling the fire is a trend to use evaluations in promotion and tenure decisions--a purpose that such assessments were not intended to have, instructors maintain. Two recent studies reveal limitations of student evaluations in fairly reflecting teaching ability. The conventional wisdom is that only smaller schools care about student course evaluations. In actuality, however, they are considered important by junior colleges, private four-year schools, state universities, the Ivy League, and top research institutions alike. At Harvard University, for example, "the biology department as a whole uses them, plus they are integral in the 20 seminar courses the department offers," says Barbara Cerva, coordinator of biology undergraduate programs. Although course evaluations have been administered on some campuses since early in the century, their use jumped in the 1960s, when student activist groups began publishing them to guide students in selecting courses. Evaluations have become even more a fixture in the 1990s, "spurred by a widespread trend toward seeking consumer satisfaction in higher education," according to Wendy Williams, an associate professor of human development at Cornell University. "As tuition has skyrocketed, parents and students have become more demanding about what takes place at college. They want control, and to be able to provide meaningful feedback. And colleges and universities are expected to listen to and respond to this feedback," she explains. A Tale Of Two Studies Williams, with Stephen Ceci, a professor of human development at Cornell, recently investigated the validity of student evaluations (W. Williams, S. Ceci, Change, 29[5]:12-23, 1997). Ceci taught developmental psychology in two consecutive semesters. The courses were identical in every way but one: the enthusiasm with which he lectured. He used a monotone in the first course and more gestures and voice inflections in the second. The differences in evaluations were startling. In the more enthusiastic semester, ratings were significantly higher for all aspects of the course, including such measures as quality of the text and suitability of the number of office hours, unrelated to lecture delivery. Yet students in the higher-rated session did not fare better academically. "If an enthusiastic delivery keeps students awake and therefore they learn more, then we'd expect their final exam scores to be superior to [exam grades of] the students who did not get an enthusiastic delivery. But this was not the case. This calls into question the validity of student ratings because they are not correlated with actual measures of learning," concludes Ceci. Another flaw in the objectivity of student evaluations is that usually the same forms are used in all classes, applying an apples-and- oranges mind-set to courses as disparate as quantum mechanics and rock music. Anthony Greenwald, a professor of psychology at the University of Washington in Seattle, and Gerald Gillmore, director of the office of educational assessment there, reviewed evaluations for 600 courses and identified a correlation between perceived course difficulty and student ratings (A. Greenwald, G. Gillmore, American Psychologist, 52:1182-6, 1209-17, November 1997). Students gave science courses, which were deemed most difficult, the lowest ratings. "Science classes tend to be harder and more demanding of effort and time," notes Gillmore. But perception of difficulty may depend on whether students are science majors. "Many science courses, especially lower-level ones, are taken by a relatively high proportion of nonmajors. That's probably relevant, but we haven't studied it directly," adds Greenwald. But Ceci's experiment on lecture style did consider student majors. Ceci found "no distinguishable difference between natural/physical science majors vs. social science majors, vs. humanity majors" in how the difficulty of his psychology course was rated. How different majors rate science courses may be revealed in an ongoing study at the University of Chicago. Jose Quintans, master of the biological sciences collegiate division, is coordinating a study that asks students to indicate their identification numbers when filling out evaluation forms, which enables researchers to correlate a major to the perceived degree of course difficulty. "This has been in effect for a year, and we don't know the answer yet," he says. HIGHER TUITION, MORE FEEDBACK: Wendy Williams, an associate professor of human development at Cornell University, says that as tuition has skyrocketed, parents and students have demanded more input as to how universities operate. Instructors look to evaluations for specific suggestions on how to improve their courses. "Students comment that they can't see what is written on the board, overheads go by too fast, they can't hear, or you talk too fast," observes Janice Glime, a professor of biological sciences at Michigan Technological University in Houghton, Mich. Evaluations told Kenneth Klemow, an associate professor of biology and geoenvironmental science at Wilkes University in Wilkes-Barre, Pa. , to use better illustrations and to explain the relevance of topics. And George Edick, a professor of biology at Rensselaer Polytechnic Institute, uses student opinions to guide lab procedure changes. "Approximately 20 percent of the policies I use in the labs have come from really good suggestions from students," he says. But sometimes students get personal. Ceci reports being taken to task for wearing orange corduroy pants, and Glime for partaking of polyester. Moreover, students with low grades have an academic ax to grind. "Students often have motives rooted in the grade they got, rather than an honest desire to help a professor improve," maintains Williams. To interpret student evaluations, some instructors use an unofficial "10-80-10 rule." Explains Klemow, "Ten percent of a class will love you, no matter how bad you are. Ten percent will hate you no matter how great you are. The remaining 80 percent are the best indicator of the class's perception." The quality of student course evaluations can determine whether a nontenure-track instructor is rehired and can play a role in tenure decisions. "Student evaluations are routinely used as fodder for promotion/tenure purposes and for reasons to let untenured faculty go in their first three years," reports Dallas Wilhelm Jr., an associate professor and chairman of the biology department at Hastings College in Hastings, Neb. James Eliason, a professor of biology at the College of Mount St. Vincent/Manhattan College in Bronx, N.Y., contends that "bad student evaluations will kill a tenure decision." But he adds that 'they are only part of the decision, however. The best evaluations in the world will not allow tenure without publications." Photo: Charles Harington DIFFERENT TACK, SAME RESULTS: Stephen Ceci taught developmental psychology two different ways in two consecutive semesters. He got higher student ratings from more enthusiastic lectures, but the higher ratings didn't translate to higher student grades. Negative reviews provide a mechanism for a department to remove an ineffective teacher by providing supposedly objective evidence. "Student evaluations are necessary because administrators tend to be real weenies about getting rid of an unproductive, irresponsible faculty member. They need input from many disgruntled 'clients'--that's what they call our students--to get rid of a faculty member without a lawsuit," declares Sandra Mitchell, an assistant professor of biology at Western Wyoming Community College in Rock Springs, Wyo. Student Ratings A student is hardly qualified to comment on whether a cell biology professor is teaching the most recently discovered signal transduction pathways. Yet course evaluation forms may ask a student to assess instructor knowledge. Many institutions are using peer review to paint a fuller picture of success in the lecture hall. In 1994, the American Association of Higher Education began a program at 16 universities called "From Idea to Prototype: The Peer Review of Teaching." The program aims to evaluate the practice of faculty observing each other's classes and providing feedback. "We view peer review as a supplement to student evaluations. It is a way for faculty to discuss their teaching in the same way that they discuss their research," explains Pamela Bender, program coordinator for peer review at the Washington, D.C.-based organization. Like student evaluations, peer review originally was intended only to provide feedback but now is used for promotion/tenure decisions, Bender adds. In addition to evaluating knowledge and currency, "peers can pay attention to nuances of teaching that they know are effective. Does the instructor stand in front of the board? Can visuals be seen from the back? Does he or she write clearly? A peer can also evaluate the crafting of the lecture, if it flows and ties into yesterday's lecture," maintains Alan Jaworski, chairman of the botany department of the University of Georgia. At Georgia, two peers review each botany course, one whose expertise is the subject area of the course, to assess accuracy, and another in a different area of botany, who views the lecture more as a student would. Peers also add perspective to the source of student complaints, such as clarifying whether a lecture is truly disorganized or the subject matter is just difficult to teach, Jaworski says. "Peer evaluations explain why students say what they do," he concludes.--R.L. If negative evaluations reflect student dissatisfaction with rigorous instructors and courses--and administrators use these evaluations in instructor retention decisions--then a natural selection of sorts may be watering down the quality of courses, a phenomenon that has been dubbed "higher education 'lite.'" "Colleges have started worrying about 'unpopular' teachers, even though the reasons may be that a professor was teaching an inherently challenging course, or giving lower grades than other instructors," contends Williams. Use of student evaluations in promotion/ tenure decisions prompts some instructors to abuse the system, claims Charlene Waggoner, a professor of biological sciences at Bowling Green State University in Bowling Green, Ohio. "They give every student who fails the first exam an automatic 'withdrawn passing'; they entertain, tell stories, and become best buddies with students; [they] inflate grades, let shoddy work slide, don't stand behind principled decisions on grading, and tell what will be on exams," she asserts. More important, she continues, is that overreliance on student ratings discourages innovation. "You take great risks when you experiment with teaching styles. The students are uncomfortable and until you learn more, you are uncomfortable. Why risk your cost-of-living raise to change your teaching?" Since student course evaluations are here to stay, instructors are trying to learn to make the most of them. And this means recognizing the evaluations' limitations, such as the fact that students cannot judge a professor's knowledge. Greenwald and Gillmore suggest two ways for administrators to design evaluation forms to make the information gathered more meaningful. First, confounding factors should be accounted for. "Student ratings of instruction can be statistically adjusted to correct for unwanted influences of grading policy (higher grades produce higher ratings), class size (larger classes get lower ratings), and other unwanted influences," they state. Second, evaluation forms should assess both productivity (how much students learn) and satisfaction with the course. TOOK ADVICE: Kenneth Klemow, an associate professor at Wilkes University, started using better illustrations and explaining the relevance of topics after student evaluations. A positive attitude on the part of instructors can help, too. Notes Ken Klemow, "Students are in our classes day after day; they must submit to our testing and grading policies. . . . We owe it to them to listen to their comments." * Ricki Lewis, a freelance science writer based in Scotia, N.Y., can be reached online at rickilewis@nasw.org.

Student Evaluations: Widespread And Controversial

Interested in reading more?

Become a Member of