All of Oregon Tech's programs conduct annual assessment of their program student learning outcomes (PSLOs) -- the key knowledge and skills that programs are designed to impart and which set students up for professional success in their discipline. In program assessment, faculty examine student work with respect to these PSLOs and ask whether the program as a whole is adequately preparing students. In areas where improvement may be called for, faculty then develop and implement plans to modify courses, curricula, or student support and eventually re-assess to confirm that changes achieved the desired results.
Program assessment is a way of making sure we honor our commitment to our students and continuously improve in the instruction we deliver in our programs. In each academic program, a program assessment coordinator leads these efforts with the collaborative involvement of all program faculty.
Changes to Assessment Processes for 2020-2021
Best Practice Example of Quality Program Assessment Report
Embedded within the Program Assessment Report Guide is Oregon Tech's program assessment report rubric. Each winter term, the Assessment Commission will evaluate program reports using this rubric and provide feedback to each program on areas to improve in the next year's assessment work.
This guide is a living document -- we welcome feedback, suggestions, and critiques as you use it.
More details will be added soon; in the meantime, see Portfolium's guide for program assessment coordinators for a brief introduction.
We will run several Assessment Days during the year for collaborative scoring of work (dates TBA), and ask that each program contribute 2-3 hours of faculty time to these days during the year. We also hope that the discussions that occur in these sessions lead to a greater sharing of ideas between faculty.
Program Assessment Report Guide
Key Deadlines for Program Assessment Coordinators
At Convocation and early in the fall term, program assessment coordinators should:
- Program Assessment:
- wrap up the previous year's assessment activity by reviewing results from the previous year and planning for improvements to be implemented in response to assessment data.
- plan for the current year's assessment activity by identifying courses or experiences where student work should be collected during the coming academic year, and setting up plans for faculty to collect and evaluate that work.
- ESLO Assessment: Work with program faculty to identify course(s) to be used for ESLO assessment
- Student Exit Survey: Work with program faculty to review and, if necessary, update your program's questions in the student exit survey.
By October 31, 2021, submit to Barb Meng (email@example.com):
- Program Assessment: Your Program Assessment Report (see Program Assessment, below)
- ESLO Assessment: Which course(s) from your program will be used for ESLO assessment (see ESLO assessment, below).
- Student Exit Survey: Any changes to your Student Exit Survey content. (see Student Exit Survey, below). If we don't hear from you, the previous year's content will be used.
Responsibilities of Program Assessment Coordinators
Program assessment coordinators coordinate program assessment activity, including:
Plan: Each year, update your program's annual PSLO assessment cycle to identify courses and assignments/activities used for program assessment and faculty responsibility for scoring and analyzing data (or for collecting data to "close the loop" and evaluate the effectiveness of improvements).
Collect: Ensure that program faculty score and analyze student work as assigned (preferably collaboratively), and that student work and scores are collected (in Portfolium or via other departmental methods).
Analyze: Ensure that data from program assessment is shared with and discussed by all program faculty. Useful questions to consider as you reflect on the results together:
Reflection on Results
- For which assessments did students meet our department’s target? For which did they not?
- What surprises us about this data?
- What does this data reveal about our students’ strengths? What does it reveal about our students’ weaknesses?
Reflection on Curriculum:
- How can we strengthen the development of our students in these outcome areas, from their first year to graduation?
- What specific changes to courses or curricula would be needed to support these outcomes better within the curriculum? (If so, who would implement them and when?)
Reflection on Assessment Process:
- Should changes be considered to our outcomes and/or how we assess them?
- Do our outcomes and the way we assess them (courses, assignment, rubrics) capture evidence of our students becoming the sort of graduates we hope to produce – both with the knowledge skills to succeed immediately post-graduation and over the course of their career?
Act: Ensure that your program/department identifies a plan of action (to improve courses, assessment processes, etc.) in response to your findings and discussion of assessment data.
Report: Ensure that the above work is documented in your annual program assessment report, due by October 31 of each year. See the Program Assessment Report Guide (above) for more details.
Coordinate ESLO assessment activity within your program, including:
- Plan: At the start of the academic year, work with program faculty to identify a course in your program to collect work for ESLO assessment. Inform the Office of Academic Excellence what course has been chosen by October 31.
- Collect: As that course is taught, help make sure program faculty upload ESLO work to Portfolium. Ensure that work is uploaded during the term the course is taught.
- Analyze: For 2020-2021, we are asking each program to contribute 2-3 hours of faculty time to assessment days for scoring and discussion of ESLO work during the academic year.
For 2020-2021, the ESLO being assessed is Inquiry and Analysis (includes problem-solving & info literacy, critical analysis & logical thinking) as well as Quantitative Literacy & Reasoning:
Definition - Inquiry and Analysis:
Inquiry and analysis consists of posing meaningful questions about situations and systems, gathering and evaluating relevant evidence, and articulating how that evidence justifies decisions and contributes to students’ understanding of how the world works
- Identify: Identify a meaningful question or topic of inquiry.
- Investigate: Critically examine existing knowledge and views on the question or topic of inquiry.
- Support: Collect evidence based on the methodology or principles of the disciplines.
- Evaluate: Critically analyze and distinguish the evidence obtained.
- Conclude: Come to a judgment based on evidence and understand the limitations and implications of that judgment.
Definition - Quantitative Literacy & Reasoning:
Quantitative literacy comprises the ability to appropriately extract, interpret, evaluate, construct, communicate, and apply quantitative information (e.g., equations, graphs, diagrams, tables, prose) and methods to solve problems, evaluate claims, and support decisions in students’ everyday professional, civic, and personal lives.
- Calculate: Perform mathematical calculations correctly and evaluate/confirm that they have done so.
- Interpret: Extract and interpret quantitative information presented in various commonly used forms.
- Construct Representations: Convert relevant quantitative information and data into different forms as appropriate.
- Apply in Context: Apply appropriate quantitative methods, draw justified conclusions, evaluate claims, and make decisions based on quantitative information. Make and evaluate key assumptions in the estimation, modeling, and data analysis.
- Communicate: In writing and (where appropriate) in speaking, effectively communicate accurate quantitative information in support of conclusions. In doing so, use representations of quantitative evidence appropriate to both audiences and purpose.
Program assessment coordinators are the primary point of contact for coordination of the programmatic portions of the Student Exit Survey.
Each year, coordinators are asked to confirm or update your program's student exit survey questions and student solicitation message. (If you don’t update this, the same questions and solicitation message will be re-used from the previous year.) Communicate any updates to the Office of Academic Excellence by October 31.
The Student Exit Survey is distributed around week six of each term based on a FAST report that pulls information from the Registrar's Office on the submission of Petition to Graduate. Summary results for each program are delivered during the summer to each program assessment coordinator and department chair.
Although not required for assessment coordinators, participating in the following opportunities will help give program assessment coordinators and other interested faculty a deeper view into the assessment process, and are likely to provide ideas to use in program assessment and course improvement: (voluntary but highly recommended)
* Program Assessment Report Review: Each year, the Assessment Commission invites all coordinators to participate in our systematic review of program assessment reports. Invitations to this will be sent out as review sessions are scheduled during winter term.
* ESLO Assessment and Engagement: ESLO committees, in coordination with Assessment Commission and the Commission on College Teaching, hold occasional workshops to support faculty in understanding ESLOs and supporting students in ESLOs. Look for these opportunities as they arise throughout the academic year.
* Assessment Days: Several periods during the academic year will be set aside for collaborative scoring and discussion of student work sampled for ESLO assessment. We ask each program to contribute several hours of faculty time to this activity over the course of the academic year, and to invite other faculty to participate as well.