OLI GRADEBOOK
Website Redesign for Student Data Visualization
MY ROLE
-
Redesigned the visuals and interactions of OLI Gradebook
-
Conducted competitive analysis, log data analysis, and contextual inquiries to identify usability issues
-
Transferred research insights into prototypes and tested with instructors to validate solutions
-
Presented research and design to the OLI director weekly
DURATION
Jun.2018 - Aug. 2018
COMPANY
Open Learning Initiative (OLI)
TOOLS
Sketch, Principle, InVision, Python, MySQL
INTRODUCTION
WHAT IS OLI GRADEBOOK?
The Open Learning Initiative (OLI) offers innovative online courses in college-level since 2008. The Gradebook in OLI website helps instructors to timely track student learning progress and performance, as well as adjust learning objectives, instruction, and assessment as necessary to drive student growth.
WHO ARE OLI USERS?
This project focused on instructors who use OLI website to teach hybrid courses (including both onsite and online learning activities) in colleges and universities.
WHY DOES IT NEED TO BE REDESIGNED?
A screenshot of old OLI Gradebook Dashboard
1
2
3
OVERWHELMING
Information
COMPLICATED
Visual Design
INFLEXIBLE
Feature Support
MISSION STATEMENT
I was expected to improve UX of Gradebook by small but effective design changes - which do not require much develop effort but can make OLI Gradebook communicate with instructors in a more efficient, flexible, and intuitive way
DESIGN PROCESS
RESEARCH
Challenge:
Too Many Good Things
After evaluating the current Gradebook dashboard, I realized that OLI has provided a lot of great features and useful information to instructors: however, it's just too much. The information architecture is not well-organized, the interface is too complicated and not intuitive, and it's too effortful for instructors to find what they should look at.
Information Ahitecture of Current OLI Gradebook
Solution:
Identify What Users Really Need
Flexible information organization
Filtering the learning data
Enable instructors to customize the Gradebook with their needs and find information more easily
Contextual Inquiries & Log Data Analysis
To understand user needs, I analyzed log data and interpreted it with interview results from contextual inquiries.
180
Days usage
1,627
Active Instructors
1,749,492
Operation records
Based on prior usage data, I identified functions that more than 50% of users used and clustered them into four main goals which supported by contextual inquiry results. I interviewed 4 instructors with more than 3-year OLI experience.
1
Check student learning progress after posting instructions or assignments and send notifications when needed
2
Gauge student learning performance and decide in-class discussion questions before lecture using Learning Dashboard/Instruction Report (linked by Gradebook)
3
Check individual student performance, and extend due date or override scores when excused
4
Export data to calculate final grades or transfer grades to Canvas/Blackboard
DESIGN INSIGHTS #1
Spotlight What Students Struggle with
When checking student learning progress after posting instructions or assignments, instructors ...
Value the bar charts of module completion which could quickly tell what's going on with a particular student as well as the whole class
Get frustrations because it's not easy to quickly find students who performed not well
Detailed view of a module when instructors expand it
DESIGN INSIGHTS #2
Reduce Load by Simplification and Shortcuts
When checking individual student performance, instructors ...
The detailed records of student performance are not clear and overwhelming
Need shortcuts to extend the due date for students or manage student lists
Although instructors can open the assessment detail page from Gradebook to override score, they cannot update due date for one student.
DESIGN INSIGHTS #3
Streamline Export and Integration with other LMS
51.3%
Users have exported OLI data based on log data analysis
4/4
Instructors in the interview used both OLI and other LMS
Instructors exported data because other LMS have more flexible grading schema as well as access to content control, and students are more comfortable with Canvas/Blackboard Gradebook. I have two options: designing new features to support the grading schema, or utilizing other systems and streamlining the integration.
I decided to improve integration between OLI and other LMS. Firstly, OLI is a small start-up and its core competence is insightful and innovative learning data analysis. The main goal of OLI Gradebook is empowering instructors to track student learning and optimizing instructions correspondingly. Therefore, it's not a grading tool. Secondly, Canvas and Blackboard have well-developed functions support grading schema, and it will take considerable yet unnecessary effort to build similar features in OLI.
4/4 instructors thought the export was PROBLEMATIC! The integration is not streamlined and in most time it requires significant reformating manually
Export data from Gradebook
When reviewing the Learning Dashboard/Instruction Report linked by Gradebook, instructors ...
Value the learning data analysis which helps instructors find which are difficult to students and make interventions
However, only heavy users utilize learning data analysis to its full potential. Light and mid users tend to stay on the Gradebook
DESIGN INSIGHTS #4
Prompt Instructors to Understand Learning Performance Deeper by Reviewing Reports
Benchmark Analysis
What can OLI Gradebook improve compared with other LMSs sharing the similar user base?
Learning data interpretation
Inform instructors about class performance at first glance
Prioritize information need to pay attention
Flexible information organization
Enable instructors to customize the Gradebook with their needs and find information more easily
ITERATION
Design Prototypes
and Iterate with Tests
I designed and iterated three prototypes in 2 weeks rapidly. Although the amount of participants in tests is small because it's not easy to recruit instructors, I generated insightful findings from each round of test and employed them as evidence to support the improvement of design
FINAL DELIVERY
Spotlight what students struggle with
Indicate low performance of students in orange and give users control to set up the color threshold of grade activities
Instructors noticed colored cells within 5 seconds and thought it was desirable
Provide insights to understand student learning performance deeper
Inform users about the baseline of learning performance to help comparison, which encourages them to review detailed reports
Novice users wanted to review report after seeing the basic statistics
Users thought it was useful when comparing individual performance with average data
Empower instructors to focus on what they value
Allow users to filter the assignment displayed
Instructors needed the Filter when they want to review typical kinds of assignments
Streamline the export process
Allow users to only export what they want, and export displayed assignments in default
“Now I don't need to download data and open it in Excel, manually delete unwanted columns, and reformat many things before uploading it to Canvas."
- from an instructor in the usability test
Re-organize and Simplified Information
Duration is more desirable than started/finished time
Number of attempts indicate enough information
New Visual Design
Keep the "old-fashion" style that instructors feel more comfortable with
Employ flat design and shadow to show information more clearly and organized
REFLECTION
Firstly, Usability
When I first opened the OLI Gradbeook, the old-fashion design style caught my attention immediately. Instead of looking closely into aesthetics and drawing new interfaces, however, I essentially started with diagnosis on current UX design to reveal usability issues, then identifying the gap between what user needs and what OLI has provided.
Navigate User Attention
Information overwhelming is a common issue in design. In this project, I took much effort in guiding users to look at the most valuable information, as well as avoiding the information to become too distracting when users want to pay attention to others. For example, in the original design I used red to indicate low performance, but finally changed it to orange because red always catches too much attention.
Design Something Tiny But Valuable
Because OLI is a successful product which has provided more than 10 years service to instructors, the website structure and design are well-developed. Therefore, I chose to improve user experience by small but valuable new design, and it's exciting to know that those tiny changes can really meet unsatisfied needs of users.
More Responsibility When Work Alone
In this independent project, I have more flexibility and freedom to play with design, and sufficient time to self-reflect on the process. More importantly, I felt I had more responsibility for the success of design - to think deeper, design better.