Live-tweet thread from the Senate discussion on CEPT

The Course Evaluation Project Team (CEPT) report was discussed (and, ultimately, endorsed) at Senate on September 18, 2017.

FAUW Board member and University of Waterloo senator Shannon Dea live-tweeted the discussion, and we’ve compiled her tweets here, with a bit of editing for readability.

UW Senate now in session. Looking forward to a lively debate on student course perception surveys starting at 4:20.

Pres. Hamullahpur reports to Senate that Pitchfork ranks UW as top university for unicorns. (It’s not as exciting as you might think.)

The President’s report is still happening. Student course perception survey discussion delayed a bit.

Ok. Here we go. Time to discuss the course evaluation project (CEPT). Here’s some background from @FAUWaterloo: [ https://twitter.com/FAUWaterloo/status/909844402142621702]

[Associate Vice-President, Academic] Mario Coniglio now giving the background on CEPT.

Coniglio describes “equity and bias [as] top of mind issues” in framing the CEPT project.

CEPT leader Mark Seasons now presenting, and again highlights negative impacts of bias on course evaluations. Says CEPT addresses this.

Seasons thanks community members, especially “three year club”, for their work on the CEPT project. Seasons reports that currently course eval practices at Waterloo vary widely across campus; some systems really outdated. Seasons provides overview of the many changes CEPT is recommending to the current system. It will evolve, the community will be updated.

George Dixon now moving endorsement of report and of Phase 2. Motion is seconded. Floor open for questions.

An amendment proposed: that the university provided sufficient resources to pilot in Fall 2017. Dixon: I haven’t seen a budget request yet.

Prez: can we come back to the amendment later? (No seconder yet.) Amendment not yet technically on the floor, but coming back.

FEDS Andrew Clubine: thanks to team. In favour of motion. Report affirms value of quality teaching esp. undergraduate. CEPT a step in the right direction, but work still to be done. An overdue improvement, but still need to address concerns, bias too. Improved data collection will help mitigate bias. Student senators will vote for motion.

FEDs prez: two key points quality teaching important and SCPs support that, but we must take concerns of bias seriously. Not a zero-sum game.

.@DavidDeVidi gives some history: FAUW members have long been concerned about annual review process. Much consultation about same. Research at UW (c. 2009) showed that good/bad teachers got same scores; that teaching evals overused in annual review. CEPT report addresses many of concerns raised in 2009. A good sign. He has received many concerned calls from faculty members about CEPT and today’s vote.

Seasons: we’ll get experts to design the tool, and then test for bias.

Bruce Richter: many senators received email from Psychology. Striking that the experts have come to a different conclusion than CEPT. Colleagues in Psych not optimistic that we’ll be able to fix bias problem. Teaching evals useful formatively to improve teaching; very different thing to affect faculty pay. Fundamentally incompatible. Tying pay to course evals at odds with supporting good teaching. Can lead profs to aim for high scores through shortcuts.

Seasons: we can’t control every aspect of bias. We’ll do the best we can. We recommended a multi-pronged approach to evaluations of teaching

Tara Collington: she and Fraser Easton received feedback from >30 Arts colleagues, prepared joint summary. Here we go: Fac members concerned about bias, unconvinced report adequately addresses concerns, concrete means for addressing bias not addressed. SWEC concerns not adequately addressed by CEPT. Validity of SCPs not adequately raised in report, discuss feedback from campus stakeholders

TC: CAUT released two recent policy statements cautioning against use of SCPs summatively. OCUFA too: SCPs should be used formatively.

TC: Collection and distribution –2 concerns. Value of data, implications of dissemination. Formative use doesn’t justify wide dissemination

Seasons: we’re aware of all of these issues. Big challenge working in multi stakeholder environment.

[Still a long speakers list. Buckle up…]

Seasons: we didn’t ignore any perspectives, but there was lots of disagreement, and we learned a lot from other universities.

Alexander Wray: students v.much in support of CEPT. Need an outlet somewhere. Otherwise, where do we go? Anecdotes of bad teaching @ Senate? What is teaching? [Time for some philosophizing… aided by some Webster’s definitions… Uh oh, and now we turn to the strat plan]

Wray: as partners, students ought to be evaluators, and ought to be participants. [whoa. Now a story from Mt. Olympus. Cautionary tale of Momus, plus some Game of Thrones. Huh]

Wray: mandate doc didn’t mention bias/equity. W2G Seasons for addressing bias. Wray: to faculty WTF guys — debate your employment conditions with your employer; don’t bother Senate with that noise.

Feridun: I’ve never seen GoT.

Gord Stubley: helped >50 faculty members read course evals, some bad comments, but also those of frustrated students trying 2b constructive. Course evals outdated and inconsistent. And our teaching has really evolved over the years. A lot! I strongly support this motion. It is an important step on the way.

[OMG too many Brians/Bryans on the speakers list. Hard to chair.] Here’s Brian/Bryan #1 [probably Brian Cepuran, alumni]: is uw prepared to invest enough to keep eValuate system running and keep the data safe?

Seasons: Yes! V. important.

Jennifer Clapp: much of what I was going to say has already been said. In answer to Wray’s question, we can’t separate tool from its purpose. Formative versus summative uses! Did CEPT discuss summative uses of the tool? Psych colleagues raised empirical concerns about this. Evidence suggests tying pay to course evals incentivizes grade inflation. Doesn’t serve faculty or students well.

GSA Prez: Robert Bruce. I’ll keep it short. Thanks for the hard work.

Bryan Tolson, @FAUWaterloo prez: thanks committee, but won’t support motion. Big concerns: biases, summative use of SCPs. What is the evidence that having a data set helps to address biases. Voting no on @FAUWaterloo board’s instructions.

[Oh, jeez. Zoned out for a minute and forgot I was live-tweeting. Ok, back now. ]

Dan O’Connor: sketches the many improvements CEPT made, endorses increasing use of other markers of teaching quality for summative purposes.

Prez: It’s getting late. Let’s keep it moving. No more long speeches. No more speakers on the list.

Mario: this initiative intended to improve teaching and learning culture on campus. Unfortunate that pol. 77 connects course evals to pay. The discussion should be around revising policy 77.

James Skidmore: lots of improvements here. We’ve long used course evals without any awareness of these issues.

Fraser Easton: thanks Seasons and team. To students: we take seriously being accountable to you. But systemic bias concern is l
egit. [Quotes colleagues seriously affected by racist/sexist comments.] Can’t continue to write a blank cheque to bias.

FEDs Hannah Beckett: formative role important. I don’t support system that reinforces bias. But we must continue to improve teaching. .

@DavidDeVidi: (1) take CAUT with a grain of salt. (2) bias also appears in research scores. Women get cited less, e.g.

And then I [@shannondea1] said: faculty can’t write CEPT a blank cheque because pol. 77 has been abused for years.

Seasons: thanks all for a lively discussion.

Motion passes. But the vote divided.

Wray is now chastizing @Fauwwaterloo and faculty again. “Talk to your employer!” he says.

New motion: to implement pilot F17, full implementation W18.

Kofi [Campbell, Renison Academic Dean]: this proposed timeline terrifies me on behalf of faculty members of colour. Change doesn’t come quickly. Let’s not rush it and botch the job. If it takes another three years to get it right, so be it.

Rob Gorbet: the process we’ve just heard about will take longer than a winter implementation.

Stubley: sympathetic to urge for a quick timeline, but cannot imagine meeting the proposed timeline.

.@JamesMSkidmore : asks Coniglio whether UW is committed to putting the resources into the project necessary to make it happen.

Coniglio: Too important to rush.

VPAP G. Dixon: We’ve taken 3 years, now have a path forward. Initiative will be adequately resourced. No need to impose artificial timelines

Motion (for Sept. timeline) soundly defeated.

Feridun: time to move forward and improve.

George Dixon: thanks Senate for fulsome discussion addressing longstanding issues.

I [@shannondea1] just moved To strike a working group to research and develop methods of assessing teaching and learning complementary to SCPs. Seconded by FEDs’ Andrew Clubine.
Chair requests tabling to next meeting. Dea and Clubine agree.

And thus ends the CEPT live-tweet. See you in the funny pages, peeps!

Senate Strategic Planning Update

David Porreca, FAUW President

This week’s post addresses issues and concerns relating to the November Senate meeting, when UW’s Strategic Plan was voted upon and approved. The UW Imprint and Daily Bulletin have already published news items relating to these events.

The discussion of the strategic plan generated a lively and vibrant debate, one which demonstrated Senate not to be a sleepy rubber-stamping body.  That said, there is some sense of disappointment in the end results. Let me explain.

The vote

Senate actually got to vote on the final version of the wording of the Strategic Plan, which had been called for at the October meeting of Senate.

Fears

  1. The fact that the Strategic Plan was printed, published and released as a public document prior to final Senate approval is, to me, the most unsettling portion of the events relating to this topic.
  2. The impact of emphasizing three specific research areas above others in the Strategic Plan is bound to colour perceptions of what research gets done at UW, including what donors see as our principal activities (e.g., will someone wanting to endow a research chair in a non-identified research area still want to send their monies to UW?)

Compromise

Concerns concentrated around the changes in wording of the Strategic Plan (detailed in our November 4 and November 11 posts) between the version Senate had approved by electronic vote in May and the one that was actually put forward at this November meeting. A friendly amendment was added to the motion to approve the Strategic Plan directing the team in charge of the implementation of the “Transformational Research” portion to refer to and take into account the university’s Strategic Research Plan, which is a much more inclusive document in terms of recognizing the broad range of research that happens on our campus. Hopefully this amendment functions to ensure the multi-million dollar internal CFI grant competition decisions are not influenced solely by the Strategic Plan statement.

Consequences

Water Institute Newsletter
Water Institute Newsletter, v. 4 issue 1, p. 5
  1. It has not taken long for evidence to arise that confirms some of our fears in relation to identifying these three research areas.  See page 5 of the Water Institute’s Fall 2013 Newsletter (pdf).
  2. After the vote was completed on the Strategic Plan itself, a motion was put forward by a colleague aiming to proclaim Senate’s own authority to govern the academic aspects of any future strategic planning at UW.  Here is the language of the motion, with amendments noted (deletions that passed during debate are struck through):

    Motion:

    Whereas the University of Waterloo Act 1972 grants the Board of Governors the power to conduct the university’s business and affairs “… save with respect to such matters as are assigned by this Act to the Senate, …”, and

    Whereas the same Act, in section 18, states that “The Senate has the power to establish the educational policies of the University … this includes the power … j. to undertake, consider and co-ordinate long-range academic planning”, and

    Whereas research and teaching are both fundamental components of the educational and academic functions of the University, and

    Whereas any strategic plan for the University will address the strategies and approaches to be taken to further education and research at the University, and will identify potential areas of opportunity or importance,

    Be it resolved that

    a. Senate asserts the powers granted to it under the Act which give Senate the final authority over all academic aspects of any Strategic Plan, including those to do with research and education.
    b. No Strategic Plan for the University shall be regarded as being in effect until an explicit motion to that effect is discussed and approved by Senate.

Debate revolved around the specifics of the wording of the motion, to the point that the eventual vote went (quite surprisingly, in my humble opinion) against the motion 30-28, with votes going along non-partisan lines. Of course, Senate does not have the authority to vote away its own authority over academic matters, which is enshrined in the UW Act. Plans are afoot with student Senators to craft a more acceptably worded motion for the January meeting of Senate to address the concerns of Senate’s role in governing the academic aspects of any future strategic planning that happens at our university.

Go Forth and Differentiate?

George Freeman, FAUW Past President

The Senate meeting of Monday, April 15 will live in my memory under the heading “kill two birds with one stone.” Unfortunately the two birds seem to have been institutional autonomy and collegial governance. Also, curiously, I don’t think I’ve ever seen more examples of the prisoner’s dilemma or tragedy of the commons illustrated in a single meeting. That’s because we had a great talk by Prof. Keith Hipel of Systems Design Engineering on methods to analyse problems which otherwise might go down those unfortunate paths. Then we went down them.

The story actually starts a long way back with a peculiar organization known as the Higher Education Quality Council of Ontario or HEQCO. In early April, it produced a report entitled “Quality: Shifting the Focus – A Report from the Expert Panel to Assess the Strategic Mandate Agreement Submissions.

Anyone wanting to understand the shortcomings of education and business leadership in Ontario must read this report, keeping in mind the credentials of the people whose names are attached to it. I promise you will be shocked, by the naked self-interest and by the lack of sound evidence-based reasoning.

The strategic mandate agreements were a bait-and-switch game foisted on university and college presidents. Under the hint of some new money, they were asked to summarize the strengths and goals of their institutions. As you might expect, there’s a strong similarity among the submissions and also a bunch of points of differentiation, such as cooperative education at Waterloo. The new money seems to have morphed into new cuts. However, differences can be amplified and marketed and that appears to excite both government and industry.

The apparent shift in “shifting the focus” is to an explicitly two-tiered system of university education (like that bastion of social equity, the United States). The big lie in “quality” is twofold in my opinion:

  1. That quality can improve under this kind of differentiation, except for a few of the privileged or lucky, and 
  2. That costs can go down.

The HEQCO report gives no evidence for any quality increase or cost decrease. What the report does indicate is that Ontario’s universities are already differentiated on everything the panel cared to measure. Yet, they repeat over and over a call to differentiate. Thus, maybe it’s useful to contemplate where the problems with university differentiation might exist, or be perceived to exist.

The main ones I could think of, off-hand, are location and faculty pay.

Location is important because Ontario has built huge settlements ringing Toronto over the past few decades but not sited enough universities within them. Also, the present universities, being close to uniformly good and close to uniformly diverse in program offerings, attract many students from their local areas. This saves a lot of money for a lot of parents (except in the huge new settlements). A recent proposal from government had three distinctly second-tier campuses being built to serve these areas. However, such a direct approach to making quality lumpy has obvious and negative bang-for-the-buck optics. In any case, Ontario’s universities are clearly too uniformly good even when badly funded.

Faculty pay is important because it’s the supposed key to a two-tiered system. The dream seems to be to have large numbers of students educated by less expensive non-research-active faculty teaching more courses per year. Let’s say on average one of these people costs $100k and teaches eight courses in comparison with a present-day faculty member who costs $130k and teaches four courses. Then, three faculty members costing $390k and teaching twelve courses could be replaced by one faculty member and one non-research-active faculty member costing $230k and teaching twelve courses.

On the surface, this would save about 40% of the 30% of the university budget which goes to faculty salaries, say twelve percent. In the university system, that’s about three to four years of inflation covered at the cost of both research diversity (including graduate courses) and research resilience dropping by 67 percent. About one third of the approximately 10% of space used for academic offices might be freed up (half a building or so at Waterloo).

This might buy enough time to fund the construction of a cheap campus or two (12% of the system budget is about $800m). It would be to the great benefit of any politicians who could announce it.
Of course, I doubt one could come anywhere near this level of transformation, so the actual potential savings must be much smaller. In any case, a system which gives a viable research opportunity to all faculty is, again, too good and too uniform to market effectively as sound bites.

The Prisoner’s Dilemma

HEQCO’s financial survival depends on telling the government what it wants to hear. They especially like surveys of public perception and arguments involving performance metrics and system-wide control by outside, non-democratic and non-participatory agencies. Increasingly, university presidents and the Council of Ontario Universities seem to just fall in line, giving up their institutional autonomy and voluntarily playing the prisoner’s dilemma game while destroying the Ontario university commons. By Feridun’s description of the last COU meeting to Senate, there is zero chance of the university presidents cooperating against differentiation. Bird number one went down but the stone continued.

On further questioning, it emerged that Waterloo would consider shutting down departments as part of differentiation; perhaps those with not enough participation in graduate studies. The next natural question was about what kind of consultation might go on before (or while) proceeding with such a differentiation agenda at Waterloo. Apparently, the meetings around the mid-cycle review of the university’s strategic plan over the past year have given the administration a mandate to go in this direction. That, and the fact that other universities are already doing it. Bird number two, dead.

One might wonder what roles Senate, its Long-Range Planning Committee, and its Finance Committee play in this, especially if you read their legal responsibilities under the University of Waterloo Act and Senate Bylaws, in particular, the Act Section 22, Bylaw 3 Section 3, and Bylaw 4 Section 3. Apparently, their roles are deemed to be none, none, and none, except that the SLRP did assist Sallie Keller (former VPAP) with the strategic mandate agreement for Waterloo and Senate still gets to ask questions. SLRP will also review, in some sense, the strategic pl
an. FAUW members have some further leverage through Articles 15, 16, and 17 of the Memorandum of Agreement. Keep in mind, however, that the province does not respect such agreements or existing legislation very much lately.

The usual message is trotted out, that Waterloo might be a net “winner” in differentiation of teaching just as we appear to be so in the differentiation of research. However, in both cases, this is just in the context of everyone losing. The United Kingdom and Germany, strangely enough, were the touted models at Senate. The UK takes performance metrics to the limits of absurdity and seems to have completely abandoned the notion of the university as a social good. Germany is distinguished by having universities which are free but mediocre, according to The Economist last June. Not exactly the leaders I would have picked to follow.

Bottom line: If we’re so innovative, why would we seek to emulate haphazard social experimentation or mediocrity?

Addendum 1: A quick scan of the just-released strategic plan draft shows “cut” only three times, as part of the phrase “cutting-edge” and, although “different” appears seven times, it is never in the context of differentiation. Whatever mandate came from the consultations isn’t obvious in the plan.

Addendum 2: HEQCO must be rolling in money because reports come too fast to read. The latest from them claims that Ontario universities are efficient, productive, and accessible – then argues for a new accountability regime to improve what we do without additional funding. OCUFA characterized it quite astutely as “fiddling at the margins.” I feel so naive having worked here for 28 years without any notion of quality to guide me.