George Bateman and Harry Roberts
For us, the following definition catches the essence of TQM:
Continually serve customers better and more economically, use
scientific methods and teamwork, and concentrate on removing all
forms of waste.
We believe that faculty can successfully adopt TQM efforts
because:
ù Professors are relatively free to change the way they
teach.
ù Professors want to be good teachers, and there are
ways--even for college presidents, deans, and department
heads--to encourage good teaching.
ù The key TQM idea is customer satisfaction and students
play the role of customers.
ù The TQM movement has already led some faculty to begin
thinking of students as customers.
The view of students as customers isn't universal; many
faculty often resist or resent this view.
The idea of students as customers can be construed much too
narrowly. Students aren't customers in the sense that the custom-
er is always right, nor are students the professors' only cus-
tomers.
But the idea of students as customers is more healthy than
the extreme paternalistic assumptions that faculty have tradi-
tionally made, namely, that professors know what's best for stu-
dents, and that students can't judge their own long-term self-
interest and have to be given a lot of medicine they don't like
to take.
This traditional professorial paternalism can lead to
complacency, stagnation, failure to check how much is really
being learned and retained, and the working hypotheses that
students' needs coincide with professors' interests. Worse, it
can lead to accepting poor student performance.
The idea of students as customers encourages professors to
take responsibility for the success of teaching, and therefore
become interested in methods of teaching improvement.
We can testify from personal experience that teaching looks
very different when you think of students as customers. Profes-
sors begin to try to figure out why students perform poorly or
challenge the relevance of the material. They begin to think
about getting relevant data.
We believe professors need more data than they usually get,
and they need it in a more timely fashion.
Role of Course Evaluations in Improving Teaching
If an institution is to achieve a customer focus, some
information about customer satisfaction is essential. Since the
late 1960s, the Graduate School of Business at the U. of Chicago,
where we teach, has used student course evaluations based on
questionnaires in all courses, with systematic public reporting
of results. Other business schools--for example, Northwestern's
Kellogg School--also have used public course evaluations.
Just as grading often makes students uncomfortable, course
evaluations make professors uncomfortable. But, in spite of minor
technical reservations, the Chicago faculty generally believe
that course evaluations provide the best available information we
have about teaching effectiveness.
Faculty members don't believe that the evaluations are mere
popularity ratings.
Although we can't prove it, we believe that teaching at
Chicago is much better than it would be in the absence of public
course evaluations because evaluations encourage the faculty to
treat students as customers, whether or not the word "customer"
is used.
In promotion decisions, a summary of course evaluations is
always included in reports and discussions of the Appointments
Committee. This, and the fact that, both at Chicago and North-
western, deans take these evaluations seriously, encourages good
teaching.
Inspirations
Unfortunately, course evaluations have limitations:
ùThey're available only after the course has ended.
ùThey use general-purpose questions that apply to all cours-
es.
ùThey can't include course-specific questions.
ùThe numerically scaled questions tell almost nothing about
what worked and what didn't.
Some information can be gleaned from tabulating free re-
sponse questions to see which themes occurred most frequently,
but these highlight pervasive problems rather than specific
difficulties.
There have been attempts at many colleges to employ simple
feedback questionnaires, typically informal but focused on the
specific class, at the end of class sessions. At Chicago, two
developments led us to experiment systematically with feedback
questionnaires and other TQM methodology:
ùIn the fall of 1990, U. of Wisconsin-Madison professor Ian
Hau was teaching a large undergraduate statistics course.
From his students, Hau formed a small quality improvement
team to help him improve the course while he was teaching
it.
ùIn March, 1991, Andrew Appel, a Chicago M.B.A. student
suggested that we use the Chicago "laboratory course" format
to help Chicago faculty members apply TQM ideas and tools to
improve their teaching, curriculum development, and re-
search.
Originally, the "laboratory course" was a "new product
laboratory" where teams of students worked with client companies
to develop and implement new product ideas. Faculty and execu-
tives from client companies coached the students. The laboratory
format has been extended to other kinds of applications such as
implementation of TQM.
The Teaching Laboratory
Thus was born "Business 712, The Laboratory to Achieve
Organizational Excellence: Improvement of Teaching, Curriculum,
and Research"--"teaching lab." In the lab, the clients are usu-
ally faculty members, and most student activity during the first
year (1991-1992) focused on helping these clients.
For example:
ù Eleven faculty members have worked with lab course stu-
dents or student teams to improve ongoing courses.
ù A team of five students worked with the behavioral science
group as a unit to design a new required course in behavior-
al science.
ù Two students worked with marketing faculty on curriculum
issues in introductory marketing courses.
ù A student worked with a faculty member to develop a course
on high-tech marketing.
ù One student in the lab has benchmarked the performance of
two of the school's most outstanding case teachers.
These efforts were generally very successful. For ongoing
courses, students developed feedback mechanisms that tell the
instructor, continually and quickly, what is and what isn't
working--both in class and in the readings--so that the instruc-
tor can make appropriate adjustments quickly.
Students used various tools, including focus groups, video-
taping, and broader surveys, but the survey tool turned out to be
a simple fast-feedback questionnaire, used at all or almost all
the class sessions. The questionnaire evolved from lengthy to
streamlined, and the process was simplified so that faculty could
do it themselves. Many of them are doing now.
They also designed simple questionnaires (often confined to
one side of one page), which faculty could administer and inter-
pret. The use of fast-feedback has become widespread, though far
from universal.
In the lab's second year, students have turned to broader
issues of curriculum development--benchmarking business efforts
in general management training--and administrative facilitation
of education--the use of information technology in M.B.A. educa-
tion.
[The Fast-Feedback Questionnaire and its Results on Teaching and
Learning]
Here we focus on what we learned developing the fast-
feedback questionnaire. Because of our experiences, we've reached
several conclusions and promising hypotheses about ways of
improving teaching.
1. It's essential for the students to be sold on the feed-
back questionnaire. Emphasizing at the start that responses will
benefit the current class, not just future ones, is key.
2. Using feedback questionnaires can hurt the instructor's
ego because sometimes there are very negative, even hostile
student reactions, even when a course is going well. However,
it's more helpful to learn about problems while one can address
them rather than encounter them on the end-of-the course ques-
tionnaire.
3. Ordinarily, instructors must rely on subjective impres-
sions as to what does and doesn't work. Our lab experiences
suggest that these impressions are often untrustworthy, and that
they tell almost nothing about variations in individual students'
or groups of students' reactions.
4. Often student feedback has suggested "obvious" problems
that weren't obvious to the instructor. For example:
ùIn almost every class there were problems hearing or under-
standing the instructor, reading the writing on the board,
or seeing the visuals.
ùAlmost always, students have wanted more examples and
applications to illustrate concepts.
ùStudents were impatient with fellow students who try to
dominate class discussions.
ùIt was very hard for the instructor to judge whether the
pace of the class is too fast or too slow, and casual stu-
dent comments weren't a reliable guide.
5. The feedback questionnaires can probe into deeper prob-
lems, such as students' understanding of basic ideas, motivation
for course preparation, or reaction to outside readings. We've
been surprised to find a common student tendency to skip readings
that have no potential impact on the course grade.
6. Probing into these deeper problems, however, requires the
instructor's intense involvement in the feedback process: s/he
must provide reverse feedback. This can be oral, written, or
both. It can take the form of course modifications, answers to
specific questions, elaboration of obscure points, clarification
of the grading system, fuller comments on student papers or
cases, additional references, or outside speakers.
7. The processes of feedback and reverse feedback tend to
draw students and instructors together to improve the learning
experience. An instructor's written reverse feedback can explain
points singled out by the fast-feedback questionnaires, and even
answer specific questions asked on the questionnaires. Reverse
feedback can require substantial time and effort of the instruc-
tor, but the payback in avoidance of rework is great.
8. Regarding reverse feedback: students want instructors to
provide feedback, preferably fast, not only on the question-
naires, but on all work they hand in. Students aren't happy with
a grade on a written assignment that doesn't include comments.
9. Course ground rules should be made explicit: students
should understand what's expected of them and what the instructor
expects to provide. It's appropriate to discuss the ground rules,
and possibly modify them with the aim of a mutual understanding-a
course "contract."
10. Instructors should devote some time to "marketing" their
courses, including the outside readings, both in advance and
during the course.
11. The fast-feedback questionnaire can discover how stu-
dents are actually using their study time. Instructors can then
use this data to help improve students' study efforts.
12. Another useful TQM aid for students may be the personal
quality checklist, developed at AT&T. This simple tool applies
TQM to personal work processes, and is adaptable to student work
processes.
13. There should be some structured instruction, even in
courses where faculty are primarily coaches and facilitators,
such as laboratory courses.
Experiences and Suggestions
In addition to facilitating and coaching students in the
lab, we've also applied some of the lessons to our own teaching
in statistics and quality management. Observing what we've
learned from the lab, we've made some immediate changes in our
own courses. We now:
ù Put a copy of the course syllabus and a short background
questionnaire into student mail folders before the first
class meeting.
ù Reduce and focus the outside readings.
ù Provide a clear idea of what each reading should accom-
plish, rather than just the general feeling that it will be
interesting or good for the students.
ù Try to "sell" the readings.
ù Use short fast-feedback questionnaires.
All these steps have proved to be helpful, but the last one
was the most important. We make no claim for novelty, because we
know of many uses of feedback questionnaires that pre-date ours.
The lab however, helped us to develop a systematic approach that
others may find useful.
The questionnaire's goal is to get systematic feedback after
every class meeting, analyze the results, and make appropriate
adjustments. Even without formal questionnaires, instructors can
actively seek out feedback on their own.
The time-honored approach to assessing one's work is to
evaluate quizzes, problems, and other assignments. Quizzes and
problems, however, often tap only limited aspects of learning; in
particular, it's often hard to tell how well students can make
the connection from abstract ideas and highly simplified examples
to real-world applications.
Since we teach mainly in the statistics area, we assign
projects that require real-world application of statistical
tools. Progress reports on these projects provide excellent feed-
back, especially on pervasive misunderstandings of statistical
ideas that students have somehow acquired before reaching our
courses.
In the spirit of the lab approach, however, instructors can
go further. They can ask questions in class, ask for a show of
hands on student experiences or problems, institute a suggestion
system, and administer--often to only a few students--very short
questionnaires, such as Mosteller's famous, "What was the muddi-
est point in this lecture?"
Course Strategy
The usefulness of feedback tools stems from the fact that
students know when they can't see or hear or are confused or
unclear about content, and can tell the instructor when a par-
ticular topic seems irrelevant to their interests.
Ideas for course strategy improvement, by contrast, must
come from the instructor--from an improved understanding of the
subject matter and its connection with other topics. For example:
ù Which topics are essential, which can be left out or
de-emphasized?
ù How can we better exploit what students already know?
ù What new topics are needed to keep the course up-to-date?
ù Are there simpler and better frameworks for understanding
the subject matter? Can one general idea unify several
specific ideas, which can then be seen as special cases of
the general idea?
ù Can process mapping and flowcharting be used to improve
course strategy?
TQM can contribute to course strategy. For example, TQM's
insistence on continual and substantial improvement is essential
to combat the tendency toward simply accepting the slow evolution
of textbooks and courses.
TQM encourages instructors to widen their horizons beyond
minor issues, such as, "Should we teach the median before we
teach the mean?" TQM tools such as benchmarking, brainstorming,
and focus groups can bring out new opportunities in course
strategy and in curriculum design.
[