Tag Archives: teaching evaluations

2018-19, Blog, Motions

US18/19-14: Phase out Current Student ‘Course Evaluations’ and Replace with Learning-Focused ‘Student Experience Surveys’

Leave a comment

Date of Notice: March 19, 2019
Current Status: April 10, 2019
Motion Type: Legislation
Sponsor: Continuous Improvement and Evaluation of Teaching Committee

Motion

Section I

1.1 WHEREAS peer-reviewed research on the student evaluation of teaching has provided evidence that student course evaluations like those used at UO reflect bias with respect to gender and may reflect bias with respect to race, and that the numerical scores are statistically orthogonal to student learning; see for example the meta-analysis indicating student ratings are not related to student learning by Uttl, White and Wong Gonzalez (2017); the randomized, controlled, blind experiment related to gender bias by Boring, Ottoboni, and Stark (2016); and an online course experiment exposing gender bias by MacNell, Driscoll, and Hunt (2015);

1.2 WHEREAS in Spring 2017 the UO Senate initiated a multi-year effort “evaluating and improving course evaluations and peer reviews with respect to reducing biases and improving validity, with the goal of improving teaching, learning, and equity” – 2017 Motion to Create Teaching Evaluation Task Force;

1.3 WHEREAS guiding principles for this work have been to ensure teaching evaluation is fair and transparent; informed by data collected from peers, students, and instructors themselves; and based on clear definitions of teaching quality rooted in the university’s broad “inclusive, engaged, and research-led” pillars; and to ensure that instructors receive actionable feedback for their own purposes of continuous improvement;

1.4 WHEREAS in Spring 2018 the UO Senate adopted a Continuous Improvement and Evaluation of Teaching System, which approved centrally administered midterm student experience surveys (results only available to instructors) and course-level instructor reflection surveys (results included in instructors’ files), and forming and charging a standing committee with developing a new end-of-term student survey and addressing other issues related to the context, policy, and tools used in teaching evaluation;

1.5 WHEREAS in January 2019 the UO Senate voted unanimously to include in instructor files language that requires teaching be evaluated “primarily using peer reviews, instructor self-reflection, and substantive written comments,” and that “numerical student evaluations of teaching should not be used as a standalone measure of teaching quality for any university purpose” – 2019 Policy on Teaching Evaluation Disclaimer Language;

1.6 WHEREAS the evaluation of courses and instructors is the work of the faculty and unit leaders, and although student feedback about their learning experience may assist evaluators in that work, a Student Experience Survey is not a course evaluation in and of itself.

Section II

2.1 THEREFORE BE IT MOVED: beginning in Fall 2019, the University will phase out its current student “Course Evaluations” and replace them with an end-of-term “Student Experience Survey” and this process will be complete by Spring 2020.

2.2 BE IT FURTHER MOVED: the Student Experience Survey will offer specific prompts for written student feedback, asking students to report how specific elements of the course—for example, course materials, feedback, organization, interactions with peers, and their own investment of time and effort—affect their learning experience. The Senate CIET Committee will develop procedures and guidelines regarding the addition of customized questions by units and/or instructors to the Student Experience Survey. The surveys will not ask generic evaluative questions such as “What is the quality of this instructor’s teaching?”

2.3 BE IT FURTHER MOVED: the information provided by the Student Experience Survey will not be used as a standalone tool to rate instructors for the purposes of merit, promotion, tenure or renewal, but rather as one source of information to be included, along with peer reviews and instructor reflections, etc., in the evaluation of instructors in light of their academic unit’s criteria for quality teaching.

2.4 BE IT FURTHER MOVED: the Senate’s Continuous Improvement and Evaluation of Teaching (CIET) Committee will continue to refine the Student Experience Survey and reporting forms in cooperation with departments, schools, and colleges. The Committee will make annual reports to the Senate about this ongoing work.

2.5 BE IT FURTHER MOVED: the Senate CIET Committee will develop procedures for determining whether to redact comments instructors have flagged as hateful or discriminatory.

2.6 BE IT FURTHER MOVED: In collaboration with the Office of Student Conduct, the Senate CIET Committee will develop procedures to ensure that students who have been found responsible for academic misconduct will be not be able to complete the Student Experience Survey for that course.

2.7 BE IT FURTHER MOVED: the Senate CIET Committee will continue to refine a protocol for instructors who wish to allow their students to complete the web-based Student Experience Survey in class. The Senate CIET Committee will also develop and disseminate materials (videos, handouts) that will help students provide feedback in the Student Experience Survey that is actionable, concrete, and focused on their learning experience.

2.8 BE IT FURTHER MOVED: the Senate CIET Committee, in collaboration with the Office of Institutional Research and the Office of the Registrar, will phase out the current numerical “P&T report” generated by the Office of the Registrar and replace it with a new annual report that combines information from the Student Experience Survey and Instructor Reflection in a format that, when combined with peer review of teaching reports, can be used by unit heads to evaluate instructors against the unit’s criteria for quality teaching. The committee will continue to study the issue of biases in student evaluations.

2.9  BE IT FURTHER MOVED: The Senate CIET Committee will produce recommendations for UO faculty search committees regarding materials they might request as evidence of teaching quality or promise above and beyond student ratings.

2017-18, Announcement, Blog

US17/18-19: Implementing a System for the Continuous Improvement and Evaluation of Teaching

Leave a comment

Date of Notice: May 2, 2018

Current Status: Approved May 23, 2018

Motion Type: Legislation

Sponsor: Bill Harbaugh & Sierra Dawson, Teaching Evaluation Task Force

Section I

1.1 Whereas: On May 24th 2017 the Senate approved motion 17/17-28 to create a task force on student evaluations with the charge of “evaluating and improving course evaluations and peer reviews with respect to reducing biases and improving validity, with the goal of improving teaching, learning, and equity.” (https://senate.uoregon.edu/entry/?Motions=US16/17-28)

1.2 Whereas: While student evaluations of teaching can be an important tool for evaluating and improving teaching and learning, there is substantial peer-reviewed evidence that student course evaluations of the sort used at UO are biased with respect to gender and race, and that the numerical scores are orthogonal to measures of teaching effectiveness and learning outcomes. (See for example Uttl, White and Gonzalez (2016) at https://www.sciencedirect.com/science/article/pii/S0191491X16300323?via%3Dihub)

1.3 Whereas: There is also scholarly evidence that, unless conducted by trained evaluators using consistent methodology, peer (faculty) reviews of teaching of the sort often done by UO departments are ineffective at evaluating teaching and do not provide useful feedback for improvement.

1.4 Whereas: While in the past UO has required that only signed written student evaluations be provided to department heads and review committees to be used in instructor review, the UO Office of General Counsel’s interpretation of the relevant law is that student classroom survey evaluations of a faculty member’s classroom or lab performance must be anonymous. (See letter from the UO General Counsel’s Office)

Section II

2.1 Therefore: The Senate directs the Committee on Committees to create and staff a Continuous Improvement and Evaluation of Teaching Committee to include faculty, graduate student, undergraduate student, and ex-officio administrative representatives, and charged with collecting feedback from stakeholders and updating the Continuous Improvement and Evaluation of Teaching System (CIETS) as needed. The 17-point chart is appended to this motion. Senate approval will be required for any changes in the CIETS except those involving the wording of questions or design of the instruments. described above.

2.2 Therefore: In Fall 2018 the committee will present to the Senate (for vote) a disclaimer for faculty personnel committees, heads, and administrators will stop using to include in reports, if those reports continue to use numerical ratings from student course evaluations in tenure and promotion reviews, merit reviews, and other personnel matters. If units or committees persist in using these numerical ratings, a statement regarding The disclaimer will address the problematic nature of those ratings, provide and an explanation for why they are being used despite those problems and include other available information regarding faculty teaching (e.g. peer reviews, student comments, faculty self-evaluations).will be included with the evaluative materials.

2.3 Therefore: As of Fall 2018 student evaluation surveys will be anonymous. Past faculty personnel committees, heads, and administrators will stop using signed written comments from previously collected student course evaluations can continue to be used in tenure and promotion reviews, merit reviews, and other personnel matters.

2.4 Therefore: Beginning with the Fall 2018 quarter, the University will stop collecting data using the current student course evaluations and will adopt the Continuous Improvement and Evaluation of Teaching System (CIETS), starting with the following elements:

  • A Midterm Student Experience Survey. This web-based anonymous survey will collect non-numerical course feedback that will be provided only to the instructor, for use in course improvement. (See spring 2018 pilot at https://oregon.qualtrics.com/jfe/form/SV_6nCcC7UHM64Etil).
  • An End-of-Term Instructor Reflection Survey. To be completed Available for optional use by the instructor and to be used for course improvement and evaluation. (See winter 2018 pilot at https://oregon.qualtrics.com/jfe/form/SV_aa47C9OFZCiFt1r; and spring pilot as .pdf).
  • An End-of-Term Student Experience Survey. This web-based anonymous survey will collect non-numerical course feedback that will be used for course improvement and evaluation.

2.5 Therefore: The committee will be charged with bringing the Senate a motion regarding the end of term student evaluation survey in Fall 2018. The current Course Evaluations will stay in place until a new version is approved by the Senate.

2.6 Therefore: The committee will be charged with bringing proposals to the Senate for vote, no later than Fall 2019, motions on the following:

Notes:

Oregon Law on classroom evaluations by students

https://www.oregonlaws.org/ors/352.226

(9) Classroom survey evaluation by students of a faculty member’s classroom or laboratory performance shall be anonymous. The record of tabulated reports shall be placed in at least one of the files designated in subsection (4) of this section. All survey instruments used to obtain evaluation data shall be returned to the faculty member.

(10) A public university listed in ORS 352.002 (Public universities) and, after July 1, 1975, but before the date on which the public university obtained a governing board, the State Board of Higher Education and its public universities, offices, departments or activities, when evaluating its employed faculty members, may not solicit or accept letters, documents or other materials, given orally or in written form, from individuals or groups who wish their identity kept anonymous or the information they provide kept confidential.

GCO letter regarding ORS 352.226 (9) and (10)

17-part chart

CONTINUOUS IMPROVEMENT AND EVALUATION OF TEACHING COMMITTEE [Tier 1]

1) Name of Committee followed by [Committee Tier Number in brackets]:

Continuous Improvement and Evaluation of Teaching Committee [Tier 1]

2) Brief Description:
The committee collects feedback on the Continuous Improvement and Evaluation of Teaching System (CIETS), makes minor changes to the instruments based on stakeholder feedback, and advises the Senate on significant changes as needed.

3) Background:
Created by Senate legislation US17/18-19: “Implementing a System for the Continuous Improvement and Evaluation of Teaching” on May 23, 2018.

4) Charge and Responsibilities:
The committee collects feedback on the Continuous Improvement and Evaluation of Teaching System (CIETS), makes minor changes to the instruments, and advises the Senate on significant changes as needed. Any proposals for Senate action during an academic year should be proposed in writing no later than the beginning of Spring Term.

5) Membership Requirements:
Three to five faculty broadly representative by college, including at least one career instructional faculty, one tenured professor, and one Senator, appointed by the Committee on Committees. One graduate student, appointed in consultation with the GSA and the GTFF. One undergraduate student appointed in consultation with the ASUO. Ex-officio members to include designees from the Office of the Provost, the Registrar, and the Teaching Engagement Program.

6) Leadership Structure (Chair, Convener &/or Staff):
a) Chair: Elected
b) Convener: Office of the Provost ex-officio member
c) Staff: Office of the Provost

7) Election of Chair (quarter, week or “at the first meeting”):
Annually at the first meeting.

8) Length of Term:
a) Non-Students (faculty, OA’s OR’s, Classified): 2 years, staggered
b) Students: 1 year
c) Ex officio: indeterminate

9) Term Limits:
a) For the Chair: none
b) For Committee Members: none
c) Ex officio: none

10) Frequency of Meetings:
At least quarterly.

11) Workload Designation:
a) For the Chair: Tier 1
b) For Committee Members: Tier 1

12) Reporting Deadline(s):
The Continuous Improvement and Evaluation of Teaching Committee shall report to the University Senate. At a minimum this report shall be in the form of an annual written report submitted by the Committee Chair to the Senate President and Senate Executive Coordinator no later than June 1. The committee shall also make additional written or oral reports to the Senate as necessary.

13) Current Members [Leave blank at present]:

14) Type:
Standing Committee

15) Category:
Academic

16) Selection Process:
Appointed

17) Additional Information:

Sample: Instructor Reflection

Sample: End-of-Term Student Experience