eLumen

Upcoming Changes

Yes, you heard right: CSP is transitioning away from eLumen to an assessment process that uses a new module in Blackboard called “Blackboard Outcomes.” We will be piloting the new system this fall using one program (BA Sport Management) and one set of institutional outcomes (Graduate School Outcomes). We do not yet have a timeline for implementation across all programs and University Outcomes. Blackboard representatives will be on campus August 15-17 working with myself, college reps, relevant program faculty, and IT staff to set up the pilot programs. I hope to have a better idea of the timeline for full implementation after that visit. I will have more updates at the Faculty Workshop in August. Until we have fully implemented the Blackboard system, we will continue to use eLumen for assessment of learning outcomes.*

How Does “Blackboard Outcomes” Work?

Blackboard Outcomes differs from eLumen in that it uses “juried assessment.” That means Blackboard pulls sample sets of artifacts (assignments submitted in Blackboard) that have been linked to the learning outcomes you’re looking to assess. Then, these samples get assigned to a group of readers – a designated group or department faculty, for example – each of whom scores each artifact in the sample according to the rubric for the outcome(s). With multiple scorers, Blackboard can provide feedback on inter-rater reliability. In this system, a sample, rather than every single assignment, can be pulled in the case of large numbers of students. Also, the course instructor is only grading the assignment for the course, not additionally assessing program outcomes. No nagging of instructors to do their eLumen scoring needed!

Implications

  • This process may require some departments to use multiple means of aggregating assessments. If not all of your program assessments are Blackboard-submissable assignments, you may need to analyze additional data from other sources. For example, test scores for tests taken in Blackboard can be aggregated and reported through Blackboard Statistics. Student performances evaluated on the basis of observations can be scored in online forms, such as Google Forms, the results of which can be aggregated and analyzed.
  • The juried assessment process will require departments to determine which faculty will be evaluators. For institutional outcomes, like Graduate School Outcomes and University Outcomes, groups of faculty will need to be formed to evaluate them.
  • Because fewer faculty will be involved in reading/scoring student artifacts, programs with large numbers of learning outcomes may not find it feasible to score all outcomes every year. These departments may want to schedule a cycle where they review student achievement of a few different outcomes each year.
  • Depending on when faculty are scoring artifacts, the traditional deadline of June 15 for Annual Reports may need to be shifted back.

*Teacher Education programs that have data used by their CAEP accreditor will continue to collect it using “Live Text,” now known as “Watermark.”

What is eLumen?

Log in to eLumen HERE

See link to eLumen in the right column on each webpage at this site! –>

Instructions for Using eLumen

 

INSTRUCTORS

Entering assessment scores into eLumen
Video instructions on how to score in eLumen
eLumen Scoring Instructions for Faculty  (PDF)

Viewing assessment results for a course
Here is how to see the aggregated results of all the scorings for a course in a term:
Viewing assessment results for a course

How will I know what to score?
Instructors will typically score students on assessments that have been determined by department faculty to be appropriate measures of the SLOs assigned to that course. Instructors will receive two notification emails from the eLumen system when they are teaching a course that has such an assessment assigned to it. Department or program chairs will inform instructors of the SLOs to be addressed in the course and the assessment used to measure them. The syllabus should reflect the SLOs and assessments used. Scoring the assessments/assignments using the rubrics in eLumen at the same time as grading it will reduce time spent and increase consistency.

DEPARTMENT/PROGRAM COORDINATORS

Checking Faculty Scoring
The department/program coordinators can check eLumen to see that scores have been entered in a timely way for assessments in their programs. Because eLumen provides an immediate link to email, communication with instructors about eLumen tasks is simple and easy for coordinators. Coordinators are responsible for making sure eLumen scoring in their programs is complete.
Here is a guide to checking that faculty have completed their eLumen scoring (PDF):
Instructions for Checking Scoring by Faculty

Scoring Reminders
Most faculty find it useful to be reminded of their eLumen scoring tasks. eLumen will automatically send an email reminder at times during the semester that you can determine. Please read the explanation below regarding eLumen email notifications.
Information on eLumen’s Automated Email Notification System

Distributing Assessments to Future Terms
Assessments in eLumen do not continue from term to term indefinitely. They need to be planned out (distributed) to future terms. You will need to make sure your program’s assessments are available and ready to be scored before the semester begins so that notifications are sent in a timely way.
Instructions for Distributing Assessments to Future Terms

Creating New Assessments
Assessments are currently set up in eLumen by Miriam Luebke or your College Assessment Coordinator. If you would like the flexibility of adding and managing assessments on your own, please contact your College Assessment Coordinator. You can review the steps involved at the link below.
Instructions for Setting Up Assessments in eLumen

Seeing Assessment Results
You should run a database report for your program/major or for University Outcomes or Graduate Outcomes at least once a year. Reports will aggregate results across courses in your department in the terms (semesters) you choose.
Generating Outcome Data Reports in eLumen

Downloading Already-Run Outcome Data Reports from eLumen’s Document Library