News From Your Board: Meeting Summary for September 27

We had two visitors to start this meeting. First, Jasmin Habib provided an update on the Course Evaluation Project Team’s implementation phase (CEPT2), in light of the recent Ryerson decision on the use of student evaluations in tenure and promotion decisions. Given that there is another project team exploring other ways of measuring teaching quality and performance (e.g., peer evaluation), CEPT2’s position, as reported at Senate on September 17, is that Waterloo is ahead of the curve and is already working to ameliorate some of the concerns raised by the decision at Ryerson. The issue of addressing bias remains contentiousThe group has nearly completed a prototype and are preparing to test it. FAUW will keep an eye on the test process to ensure it doesn’t disadvantage any vulnerable faculty.

Sebastian Siebel-Achenbach then joined us to give an update from the OCUFA (Ontario Confederation of University Faculty Associations) Contract Faculty and Faculty Complement Committee. He wants to raise some awareness and support for CAUT’s upcoming Fair Employment Week and OCUFA’s Fairness for Contract Faculty campaign, in light of the growing informalization of teaching and other kinds of professorial work in Canada. So check those links out.

We had a short discussion about the timeline and communication around proposed changes to our dues structure. These changes, if adopted by the membership, will bring FAUW in line with the conventions of other faculty associations, and alleviate some of the existing inequalities in the existing structure. More information about the proposal is available on our website. Voting members will receive an email with the details next week and a link to the online ballot on October 15. Voting will be open from October 15 to 19.

We spent some time reviewing responses from our members to the Faith Goldy event that did not go ahead earlier this year, in light of the Ontario government’s recent mandate that universities issue a free speech policy. Most responses supported our position on the event, which was issued on April 23rd and emphasized the association’s support of immigrant and non-Canadian members.

The Board appointed Mathieu Doucet as the new FAUW representative on the University Advisory Committee on Traffic Violations and Parking. We would like to see the committee expand its mandate to include active transportation and are confident that Mathieu will be a passionate advocate for this.

We ended, as always, with a review of upcoming events, including the campus tour on Thursday, the Council of Representatives meeting on October 17,  a workshop on university governance November 9, and a talk by Mary Hardy on November 16 about the joint university pension plan being developed for University of Toronto, University of Guelph, and Queen’s University employees.

News From Your Board: Meeting Summary for September 13, 2018

Daniel Cockayne (Geography and Environmental Management) and Brent Matheson (Math/Business and Accounting) are taking over the Board meeting recaps this year. Here’s Daniel’s first go: 

The first FAUW Board meeting of the 2018/2019 academic year (meet your new Board members here!) began with a summary of FAUW’s finances by Michelle Adams from the accountancy firm RLB, which audits FAUW’s finances annually. You’ll be glad to hear that she confirmed that FAUW’s financial situation is sound!

Long in the works, there is to be a new policy that will establish a clear process for dealing with accommodations for faculty and staff. As an FS-class policy, it will need to be approved by the faculty and staff relations committees (FRC and SRC). The terms of reference for the policy have been drafted, and a committee will be appointed soon.

We discussed the final details of the agreement between FAUW and the Renison Association of Academic Staff (RAAS). Pending adoption by both the RAAS and FAUW memberships, this agreement will make RAAS members affiliate members of FAUW. They will pay FAUW dues in exchange for access to specific services and resources. We expect to bring this to the membership at the Fall General Meeting on December 4.

The Board approved new election procedures, now posted on our website, which mostly just document existing practices and will be reflected in the next call for nominations for FAUW President this fall.

We discussed the current re-development of student perception surveys at Waterloo in light of the arbitration decision at Ryerson University, which ruled against the use of student surveys in tenure and promotion decisions. This has established an important precedent for faculty associations and universities across Canada. It is important to emphasize that this decision does not dismiss the value of student surveys wholesale, just their use for summative decisions pertaining to merit, tenure, and promotion. FAUW discussed how this decision affects (or should affect) Waterloo’s current Course Evaluation Project Team 2 (CEPT2) and Complimentary Teaching Assessment Project Team (CTAPT). (See also: Jay Michela’s recent analysis of this issue and all our blog posts about student surveys.)

Another point of discussion was FAUW’s response to the recent announcement from the Office of the Premiere that publicly-assisted universities and colleges must post a policy on free speech by January 1, 2019. There were questions around precisely how formalized such a policy needs to be and the kinds of protections for free speech that the University of Waterloo already has in place. There will certainly be more to come on this topic. Stay tuned.

Feedback on Feedback Questionnaires’ Use and Misuse

Jay Michela addresses misconceptions in Alex Usher’s analysis of the Ryerson arbitration decision. 

Guest post by Jay Michela, Psychology. 

Alex Usher of Higher Education Strategy Associates (HESA) has offered his analysis of an arbitration decision at Ryerson University which ruled against conventional use of students’ course ratings for personnel decisions (tenure and promotion decisions). It has been circulated within our university and elsewhere (e.g., to OCUFA), and appears on the HESA website under the headline “Time to Talk Teaching Assessments.”

I was moved to respond to Usher’s statement because it expresses many of the misconceptions that exist around summative use of students’ ratings of courses and instructors.

What follows is the full text of Alex Usher’s analysis, with my responses interspersed. I hope this format for explaining the urgent need to change university practices around student questionnaires turns out to be more engaging and pithy than some of the literature reviews and other research reports on which this material is based. Continue reading “Feedback on Feedback Questionnaires’ Use and Misuse”

Ryerson Arbitration Decision Regarding Use of Student Evaluations of Teaching

A decision has come in regarding the use of Student Evaluations of Teaching at Ryerson University.

Below is a summary excerpted from the Ontario Confederation of University Faculty Associations (OCUFA) blog.

Read the full decision: Ryerson University v Ryerson Faculty Association, 2018 CanLII 58446 (ON LA) Continue reading “Ryerson Arbitration Decision Regarding Use of Student Evaluations of Teaching”

News From Your Board: February 1 Meeting Recap

—Peter Johnson, director for the Faculty of Environment

This mid-winter Board meeting kicked off with a discussion with the negotiating team about the details of the memorandum of salary settlement. As you have no doubt seen, this settlement shows evidence of the strong productive and collegial relationship between FAUW and the administration, and sets a positive foundation for future salary settlements. A well-deserved note of appreciation to the entire negotiating team for their tireless work on our collective behalf.

In addition to concluding salary negotiations, we are now entering a busy FAUW events season, with many exciting workshops, panels, and meetings over the remainder of the winter term. Of particular note are the upcoming Hagey Lecture, workshops on Navigating University Governance and Writing University Policy, the President’s Luncheon on Academic Freedom, and, looking further ahead, the Spring General Meeting and Tenure & Promotion workshops in April.

Ongoing topics included exam scheduling, specifically discussions with the Registrar’s Office to better understand the current timing of the release of the exam schedule, and the possibility of moving it earlier in the term. We feel that early release of this schedule will be a positive support for both student and instructor mental health.

The recently revealed ‘bug’ in Evaluate, the online course evaluation software used by the University to collect student course perceptions (which are in turn used to evaluate faculty for merit pay), caused much discussion. Though we applaud how the discovery of the bug was handled, there is much work to be done to develop a system of governance to guide the Evaluate project. We look forward to discussing how Evaluate will be used and its governance structure with the University and IST in the future.

News From Your Board – September 28 Meeting

—Sally Gunz, past president

Peter Johnson and I agreed to take turns writing recaps of FAUW board meetings. We’ll focus on the main items and try to avoid the chit-chat that make all board meetings memorable events. For a fee, we will provide the chit-chat separately!

One of the lingering issues from the recent Memorandum of Agreement (MoA) revisions is that of histograms (13.5.11). This has proven to be technically challenging and a sub-group of the Faculty Relations Committee has been working on revisions for some time. The sub-group continues to work on this and will be providing recommendations soon.

FAUW senators summarized the debate on the CEPT report at Senate. A motion will be moved at the next senate meeting to create a new working group that will develop appropriate methodology to assess teaching per se (and not student perceptions of teaching, which is the focus of the current evaluations). You can read more about this motion on the FAUW website.

Bryan Tolson brought the board up to speed on recent changes to the Office of General Counsel and there was general agreement that these were positive for our membership. We also conducted a sound debrief of our new faculty events.

We discussed the impact of Unit4 on research accounts and the difficulty in gathering information. Peter Johnson indicated that as a stopgap measure, IST developed a tool to query and export salary reports from Unit4. This tool is generally easy to use and would be a significant advantage if it were extended to all areas of research grant reporting. Ultimately, while Unit4 may meet the needs of Finance as-is, it does not meet the needs of admin or faculty. This alternate tool developed in-house and ably supported by IST has the ability to do so. Discussions at UCIST (University Committee on Information Systems and Technology) to this effect will continue.

Live-tweet thread from the Senate discussion on CEPT

The Course Evaluation Project Team (CEPT) report was discussed (and, ultimately, endorsed) at Senate on September 18, 2017.

FAUW Board member and University of Waterloo senator Shannon Dea live-tweeted the discussion, and we’ve compiled her tweets here, with a bit of editing for readability.

UW Senate now in session. Looking forward to a lively debate on student course perception surveys starting at 4:20.

Pres. Hamullahpur reports to Senate that Pitchfork ranks UW as top university for unicorns. (It’s not as exciting as you might think.)

The President’s report is still happening. Student course perception survey discussion delayed a bit.

Ok. Here we go. Time to discuss the course evaluation project (CEPT). Here’s some background from @FAUWaterloo: []

[Associate Vice-President, Academic] Mario Coniglio now giving the background on CEPT.

Coniglio describes “equity and bias [as] top of mind issues” in framing the CEPT project.

CEPT leader Mark Seasons now presenting, and again highlights negative impacts of bias on course evaluations. Says CEPT addresses this.

Seasons thanks community members, especially “three year club”, for their work on the CEPT project. Seasons reports that currently course eval practices at Waterloo vary widely across campus; some systems really outdated. Seasons provides overview of the many changes CEPT is recommending to the current system. It will evolve, the community will be updated.

George Dixon now moving endorsement of report and of Phase 2. Motion is seconded. Floor open for questions.

An amendment proposed: that the university provided sufficient resources to pilot in Fall 2017. Dixon: I haven’t seen a budget request yet.

Prez: can we come back to the amendment later? (No seconder yet.) Amendment not yet technically on the floor, but coming back.

FEDS Andrew Clubine: thanks to team. In favour of motion. Report affirms value of quality teaching esp. undergraduate. CEPT a step in the right direction, but work still to be done. An overdue improvement, but still need to address concerns, bias too. Improved data collection will help mitigate bias. Student senators will vote for motion.

FEDs prez: two key points quality teaching important and SCPs support that, but we must take concerns of bias seriously. Not a zero-sum game.

.@DavidDeVidi gives some history: FAUW members have long been concerned about annual review process. Much consultation about same. Research at UW (c. 2009) showed that good/bad teachers got same scores; that teaching evals overused in annual review. CEPT report addresses many of concerns raised in 2009. A good sign. He has received many concerned calls from faculty members about CEPT and today’s vote.

Seasons: we’ll get experts to design the tool, and then test for bias.

Bruce Richter: many senators received email from Psychology. Striking that the experts have come to a different conclusion than CEPT. Colleagues in Psych not optimistic that we’ll be able to fix bias problem. Teaching evals useful formatively to improve teaching; very different thing to affect faculty pay. Fundamentally incompatible. Tying pay to course evals at odds with supporting good teaching. Can lead profs to aim for high scores through shortcuts.

Seasons: we can’t control every aspect of bias. We’ll do the best we can. We recommended a multi-pronged approach to evaluations of teaching

Tara Collington: she and Fraser Easton received feedback from >30 Arts colleagues, prepared joint summary. Here we go: Fac members concerned about bias, unconvinced report adequately addresses concerns, concrete means for addressing bias not addressed. SWEC concerns not adequately addressed by CEPT. Validity of SCPs not adequately raised in report, discuss feedback from campus stakeholders

TC: CAUT released two recent policy statements cautioning against use of SCPs summatively. OCUFA too: SCPs should be used formatively.

TC: Collection and distribution –2 concerns. Value of data, implications of dissemination. Formative use doesn’t justify wide dissemination

Seasons: we’re aware of all of these issues. Big challenge working in multi stakeholder environment.

[Still a long speakers list. Buckle up…]

Seasons: we didn’t ignore any perspectives, but there was lots of disagreement, and we learned a lot from other universities.

Alexander Wray: students v.much in support of CEPT. Need an outlet somewhere. Otherwise, where do we go? Anecdotes of bad teaching @ Senate? What is teaching? [Time for some philosophizing… aided by some Webster’s definitions… Uh oh, and now we turn to the strat plan]

Wray: as partners, students ought to be evaluators, and ought to be participants. [whoa. Now a story from Mt. Olympus. Cautionary tale of Momus, plus some Game of Thrones. Huh]

Wray: mandate doc didn’t mention bias/equity. W2G Seasons for addressing bias. Wray: to faculty WTF guys — debate your employment conditions with your employer; don’t bother Senate with that noise.

Feridun: I’ve never seen GoT.

Gord Stubley: helped >50 faculty members read course evals, some bad comments, but also those of frustrated students trying 2b constructive. Course evals outdated and inconsistent. And our teaching has really evolved over the years. A lot! I strongly support this motion. It is an important step on the way.

[OMG too many Brians/Bryans on the speakers list. Hard to chair.] Here’s Brian/Bryan #1 [probably Brian Cepuran, alumni]: is uw prepared to invest enough to keep eValuate system running and keep the data safe?

Seasons: Yes! V. important.

Jennifer Clapp: much of what I was going to say has already been said. In answer to Wray’s question, we can’t separate tool from its purpose. Formative versus summative uses! Did CEPT discuss summative uses of the tool? Psych colleagues raised empirical concerns about this. Evidence suggests tying pay to course evals incentivizes grade inflation. Doesn’t serve faculty or students well.

GSA Prez: Robert Bruce. I’ll keep it short. Thanks for the hard work.

Bryan Tolson, @FAUWaterloo prez: thanks committee, but won’t support motion. Big concerns: biases, summative use of SCPs. What is the evidence that having a data set helps to address biases. Voting no on @FAUWaterloo board’s instructions.

[Oh, jeez. Zoned out for a minute and forgot I was live-tweeting. Ok, back now. ]

Dan O’Connor: sketches the many improvements CEPT made, endorses increasing use of other markers of teaching quality for summative purposes.

Prez: It’s getting late. Let’s keep it moving. No more long speeches. No more speakers on the list.

Mario: this initiative intended to improve teaching and learning culture on campus. Unfortunate that pol. 77 connects course evals to pay. The discussion should be around revising policy 77.

James Skidmore: lots of improvements here. We’ve long used course evals without any awareness of these issues.

Fraser Easton: thanks Seasons and team. To students: we take seriously being accountable to you. But systemic bias concern is l
egit. [Quotes colleagues seriously affected by racist/sexist comments.] Can’t continue to write a blank cheque to bias.

FEDs Hannah Beckett: formative role important. I don’t support system that reinforces bias. But we must continue to improve teaching. .

@DavidDeVidi: (1) take CAUT with a grain of salt. (2) bias also appears in research scores. Women get cited less, e.g.

And then I [@shannondea1] said: faculty can’t write CEPT a blank cheque because pol. 77 has been abused for years.

Seasons: thanks all for a lively discussion.

Motion passes. But the vote divided.

Wray is now chastizing @Fauwwaterloo and faculty again. “Talk to your employer!” he says.

New motion: to implement pilot F17, full implementation W18.

Kofi [Campbell, Renison Academic Dean]: this proposed timeline terrifies me on behalf of faculty members of colour. Change doesn’t come quickly. Let’s not rush it and botch the job. If it takes another three years to get it right, so be it.

Rob Gorbet: the process we’ve just heard about will take longer than a winter implementation.

Stubley: sympathetic to urge for a quick timeline, but cannot imagine meeting the proposed timeline.

.@JamesMSkidmore : asks Coniglio whether UW is committed to putting the resources into the project necessary to make it happen.

Coniglio: Too important to rush.

VPAP G. Dixon: We’ve taken 3 years, now have a path forward. Initiative will be adequately resourced. No need to impose artificial timelines

Motion (for Sept. timeline) soundly defeated.

Feridun: time to move forward and improve.

George Dixon: thanks Senate for fulsome discussion addressing longstanding issues.

I [@shannondea1] just moved To strike a working group to research and develop methods of assessing teaching and learning complementary to SCPs. Seconded by FEDs’ Andrew Clubine.
Chair requests tabling to next meeting. Dea and Clubine agree.

And thus ends the CEPT live-tweet. See you in the funny pages, peeps!