The Course Evaluation Project Team (CEPT) report was discussed (and, ultimately, endorsed) at Senate on September 18, 2017.
FAUW Board member and University of Waterloo senator Shannon Dea live-tweeted the discussion, and we’ve compiled her tweets here, with a bit of editing for readability.
UW Senate now in session. Looking forward to a lively debate on student course perception surveys starting at 4:20.
Pres. Hamullahpur reports to Senate that Pitchfork ranks UW as top university for unicorns. (It’s not as exciting as you might think.)
The President’s report is still happening. Student course perception survey discussion delayed a bit.
Ok. Here we go. Time to discuss the course evaluation project (CEPT). Here’s some background from @FAUWaterloo: [ https://twitter.com/FAUWaterloo/status/909844402142621702]
[Associate Vice-President, Academic] Mario Coniglio now giving the background on CEPT.
Coniglio describes “equity and bias [as] top of mind issues” in framing the CEPT project.
CEPT leader Mark Seasons now presenting, and again highlights negative impacts of bias on course evaluations. Says CEPT addresses this.
Seasons thanks community members, especially “three year club”, for their work on the CEPT project. Seasons reports that currently course eval practices at Waterloo vary widely across campus; some systems really outdated. Seasons provides overview of the many changes CEPT is recommending to the current system. It will evolve, the community will be updated.
George Dixon now moving endorsement of report and of Phase 2. Motion is seconded. Floor open for questions.
An amendment proposed: that the university provided sufficient resources to pilot in Fall 2017. Dixon: I haven’t seen a budget request yet.
Prez: can we come back to the amendment later? (No seconder yet.) Amendment not yet technically on the floor, but coming back.
FEDS Andrew Clubine: thanks to team. In favour of motion. Report affirms value of quality teaching esp. undergraduate. CEPT a step in the right direction, but work still to be done. An overdue improvement, but still need to address concerns, bias too. Improved data collection will help mitigate bias. Student senators will vote for motion.
FEDs prez: two key points quality teaching important and SCPs support that, but we must take concerns of bias seriously. Not a zero-sum game.
.@DavidDeVidi gives some history: FAUW members have long been concerned about annual review process. Much consultation about same. Research at UW (c. 2009) showed that good/bad teachers got same scores; that teaching evals overused in annual review. CEPT report addresses many of concerns raised in 2009. A good sign. He has received many concerned calls from faculty members about CEPT and today’s vote.
Seasons: we’ll get experts to design the tool, and then test for bias.
Bruce Richter: many senators received email from Psychology. Striking that the experts have come to a different conclusion than CEPT. Colleagues in Psych not optimistic that we’ll be able to fix bias problem. Teaching evals useful formatively to improve teaching; very different thing to affect faculty pay. Fundamentally incompatible. Tying pay to course evals at odds with supporting good teaching. Can lead profs to aim for high scores through shortcuts.
Seasons: we can’t control every aspect of bias. We’ll do the best we can. We recommended a multi-pronged approach to evaluations of teaching
Tara Collington: she and Fraser Easton received feedback from >30 Arts colleagues, prepared joint summary. Here we go: Fac members concerned about bias, unconvinced report adequately addresses concerns, concrete means for addressing bias not addressed. SWEC concerns not adequately addressed by CEPT. Validity of SCPs not adequately raised in report, discuss feedback from campus stakeholders
TC: CAUT released two recent policy statements cautioning against use of SCPs summatively. OCUFA too: SCPs should be used formatively.
TC: Collection and distribution –2 concerns. Value of data, implications of dissemination. Formative use doesn’t justify wide dissemination
Seasons: we’re aware of all of these issues. Big challenge working in multi stakeholder environment.
[Still a long speakers list. Buckle up…]
Seasons: we didn’t ignore any perspectives, but there was lots of disagreement, and we learned a lot from other universities.
Alexander Wray: students v.much in support of CEPT. Need an outlet somewhere. Otherwise, where do we go? Anecdotes of bad teaching @ Senate? What is teaching? [Time for some philosophizing… aided by some Webster’s definitions… Uh oh, and now we turn to the strat plan]
Wray: as partners, students ought to be evaluators, and ought to be participants. [whoa. Now a story from Mt. Olympus. Cautionary tale of Momus, plus some Game of Thrones. Huh]
Wray: mandate doc didn’t mention bias/equity. W2G Seasons for addressing bias. Wray: to faculty WTF guys — debate your employment conditions with your employer; don’t bother Senate with that noise.
Feridun: I’ve never seen GoT.
Gord Stubley: helped >50 faculty members read course evals, some bad comments, but also those of frustrated students trying 2b constructive. Course evals outdated and inconsistent. And our teaching has really evolved over the years. A lot! I strongly support this motion. It is an important step on the way.
[OMG too many Brians/Bryans on the speakers list. Hard to chair.] Here’s Brian/Bryan #1 [probably Brian Cepuran, alumni]: is uw prepared to invest enough to keep eValuate system running and keep the data safe?
Seasons: Yes! V. important.
Jennifer Clapp: much of what I was going to say has already been said. In answer to Wray’s question, we can’t separate tool from its purpose. Formative versus summative uses! Did CEPT discuss summative uses of the tool? Psych colleagues raised empirical concerns about this. Evidence suggests tying pay to course evals incentivizes grade inflation. Doesn’t serve faculty or students well.
GSA Prez: Robert Bruce. I’ll keep it short. Thanks for the hard work.
Bryan Tolson, @FAUWaterloo prez: thanks committee, but won’t support motion. Big concerns: biases, summative use of SCPs. What is the evidence that having a data set helps to address biases. Voting no on @FAUWaterloo board’s instructions.
[Oh, jeez. Zoned out for a minute and forgot I was live-tweeting. Ok, back now. ]
Dan O’Connor: sketches the many improvements CEPT made, endorses increasing use of other markers of teaching quality for summative purposes.
Prez: It’s getting late. Let’s keep it moving. No more long speeches. No more speakers on the list.
Mario: this initiative intended to improve teaching and learning culture on campus. Unfortunate that pol. 77 connects course evals to pay. The discussion should be around revising policy 77.
James Skidmore: lots of improvements here. We’ve long used course evals without any awareness of these issues.
Fraser Easton: thanks Seasons and team. To students: we take seriously being accountable to you. But systemic bias concern is l
egit. [Quotes colleagues seriously affected by racist/sexist comments.] Can’t continue to write a blank cheque to bias.
FEDs Hannah Beckett: formative role important. I don’t support system that reinforces bias. But we must continue to improve teaching. .
@DavidDeVidi: (1) take CAUT with a grain of salt. (2) bias also appears in research scores. Women get cited less, e.g.
And then I [@shannondea1] said: faculty can’t write CEPT a blank cheque because pol. 77 has been abused for years.
Seasons: thanks all for a lively discussion.
Motion passes. But the vote divided.
Wray is now chastizing @Fauwwaterloo and faculty again. “Talk to your employer!” he says.
New motion: to implement pilot F17, full implementation W18.
Kofi [Campbell, Renison Academic Dean]: this proposed timeline terrifies me on behalf of faculty members of colour. Change doesn’t come quickly. Let’s not rush it and botch the job. If it takes another three years to get it right, so be it.
Rob Gorbet: the process we’ve just heard about will take longer than a winter implementation.
Stubley: sympathetic to urge for a quick timeline, but cannot imagine meeting the proposed timeline.
.@JamesMSkidmore : asks Coniglio whether UW is committed to putting the resources into the project necessary to make it happen.
Coniglio: Too important to rush.
VPAP G. Dixon: We’ve taken 3 years, now have a path forward. Initiative will be adequately resourced. No need to impose artificial timelines
Motion (for Sept. timeline) soundly defeated.
Feridun: time to move forward and improve.
George Dixon: thanks Senate for fulsome discussion addressing longstanding issues.
I [@shannondea1] just moved To strike a working group to research and develop methods of assessing teaching and learning complementary to SCPs. Seconded by FEDs’ Andrew Clubine.
Chair requests tabling to next meeting. Dea and Clubine agree.
And thus ends the CEPT live-tweet. See you in the funny pages, peeps!
The FAUW Blog 