Zoom Case Study: Improving Classroom Communication

Elton Ho, Jon Carlo Bibat, Zilu Zhu

Background

What are we improving?

We are aiming to improve functionality on Zoom, specifically the “Participants” window section.

Who’s our target audience?

Our target audience for this case study will be K-12 students and teachers, with a focus on middle school and high school.

Introduction

Through our own personal experiences as well as the many frustrations aired out on social media, we know how difficult it is to find even the motivation to attend or just concentrate in a virtual class. On top of this already significant dilemma, the current design of applications used for online classes like Zoom doesn’t exactly mimic the layout of a traditional classroom environment where teachers have a direct view over all of their students, and therefore have the ability to read their facial expressions or body language. This makes it difficult for teachers to gauge the understanding of their students and for students to engage with the material being presented. To combat this and improve communication between teachers and their students during class time, we want to allow for students to signal better to their teacher if the class is confused or if the material is not covered in depth. We believe the current functionality of this feature (raise hand, yes, no, go slower etc.) is not widely used due to the fact that teachers won’t see the Zoom participants window when they are screen sharing lecture slides. By addressing this specific issue with virtual classrooms, we hope to significantly augment and improve the online learning environment by increasing interactivity from both sides.

Today’s minds are tomorrow’s leaders. Knowing this, we want to emphasize ways in which we can better foster their education and growth so that they can reach their full potential.

Research

We conducted user research to get a better grasp at what is the current state of K-12 remote education. We surveyed K-12 students and teachers asking for their experiences with Zoom. We in particular wanted to know how often students and teachers use the Participants window, as there are options to display one’s status. In addition, we also directly observed those of us who had siblings in middle school and high school attending class online.

Survey Results

Our users are neutral and somewhat satisfied towards Zoom, though the ones who are unhappy bring up good points to explain their dissatisfaction. We found that approximately half of the students do not use the participants window and the ones who do use it very little or only when teachers request them to. Most do not believe there are any features that can be improved regarding student participation, but 10/14 students report things that could be improved for online education. Overall, the findings are inconclusive but there are insightful quotes from users who feel strongly about the subject.

Regarding the Participants window:

“My teacher’s barely use it because it confuses them. I don’t know what exactly it is but maybe they should get tutorials or something.”

“I’ve never used these features unless the teacher asked to.”

Regarding student-to-teacher and teacher-to-student interactions:

“Yes, [you] cannot get help as easily, [it’s] harder to explain things, and no-one talks in class besides teacher.”

“Yes. It’s hard to reinforce things because the psychology between the student and the teacher is no longer there. I can’t gauge if my students are understanding the material without seeing their work or interaction.”

Figure 1. Chart showing satisfaction with online education

Figure 2. Chart showing engagement with 1 = “Not Engaged At All” & 7 = “Very Engaged”

When asked whether they were satisfied with the quality of online education, a majority of students were either indifferent or dissatisfied with it. The distribution of these answers almost directly correlated with the responses to the question about engagement during class. This indicates that a majority of the students found themselves only paying attention to what was being presented in class about half the time. From this data, we might be able to assume that the dissatisfaction with the quality of learning might contribute to the lack of engagement of class as many would not want to invest much time or attention into something that just doesn’t project the same quality as they were used to.

Direct Observation Results

From our direct observations, we found that there is a common theme of distraction during online learning. The students would often engage in other activities that are outside of the sphere of learning.

Participant 1 — Elvis

Figure 3. Elvis is watching a lecture on full screen on Zoom (left). Elvis switched from Zoom to discord, where he replies to a message (center). Elvis looks at the class Piazza during lecture, possibly to clarify confusion on lecture content (right).

• Elvis occasionally stares at his phone. It is placed in front of the computer under his chin.
• He cycles through a variety of class forums and portals during class time, looking at Piazza, a code portal, the class webpage/portal, and the Zoom lecture.
• Outside these online education tools that Elvis looks at, he also goes on Discord to message other people.
• Elvis also views the Zoom class lecture in full screen mode, meaning that the participants window and Zoom messaging window are separate and floating. He actually did not open the participants window at all during my period of observation.

Participant 2 — Jazmine Kay

Figure 4. During class, Jazmine has several distractions (phone and food) that keep her from consistently paying attention.

• While teacher presents, Jazmine Kay pays attention to her teacher, but often engages with her friend who is currently Facetiming her or eating her breakfast whenever the teacher isn’t directly addressing the class or asking questions.
• When put into breakout rooms with her peers, neither she nor the students interact with each other and do not focus on what the teacher told them to do. All of the students engage in other activities, such as playing games on their phones or texting friends. During this time, Jazmine Kay mainly engages in the activities she participated in when her teacher wasn’t addressing the class (eating or talking to her friend on Facetime).
• When returned to the main session, she did the same thing as before.
• Whenever the teacher asked questions, the students had videos on and mics muted and either responded verbally or remained distracted; it’s assumed that the teacher just inferred that her students understood or just read the mouths of her muted students.
• At no point in time was the participants window ever used or engaged with in this observation session.

Problem Statement

Students need a way to communicate feedback-sharing in an online learning environment so that teachers can adapt their teaching methods in real time to improve student comprehension.

User Personas

From the results of our user research and problem statement, we were then able to create user personas that are our target demographics for this case study:

Margo Mannheim-Mannheim (secondary group persona — teachers)

Age: 35

Background: Eccentric, God-Fearing, Teacher-esque

Margo is a high school calculus teacher teaching at Elefino High School. Margo has trouble with keeping her class motivated during her lessons, and she finds it difficult to gauge at the level of comprehension in her classes. When grading homework assignments, she almost always finds most of them at B’s and C’s, underscoring the lack of command her students have on the calculus material. Margo also occasionally gets emails from her students asking for more help on the homework assignments. She tries her best to help every student, but she’s also frustrated in having to re explain concepts over and over again each time.

Motivations:

• Better understand the level of understanding of the class for the course material.
• Reduce the need for students to have to ask for help by increasing course comprehension.
• Raise the class averages for homework assignments.

Frustrations:

• Her students rarely ask clarification questions in class, despite the large amounts of emails she receives from her students asking for help with the course material.

Holly Hockman

Age: 17

Background: Driven, Hard-working, (More Recently) Jaded

Hollyhock is a junior attending Elefino High School. Since the start of the pandemic, she has had a hard time finding the motivation to do a lot of her school work because her house has now transformed into her school environment as well as her home environment. Her entire dynamic and workflow has been completely overturned by this change. Before, she was a focused and highly driven student, who aimed to constantly be at the top of her class. Now, during class, she’s distracted by the Internet, her phone, and her family members. Additionally, whenever she tries to answer or ask a question during class by raising her hand, she’s almost always unseen or ignored by the instructor which adds on to the lack of motivation to pay attention during class as she knows she’s never going to be asked to do anything; she and her peers just log on to listen to a lecture and log off. Overall, she is no longer as motivated by or focused on school due to a lack of interactions between her and her instructors, which is concerning as it is the most important year of her high school career.

Motivations:

• Stay at the top her class and pass all of her exams to get into her dream school
• Find a system to stay motivated and focused for school during class and outside of class despite being at home all day, everyday

Frustrations:

• Not being able to interact with her instructors as she did before the emergence of the virtual learning environment
• Not being able to stay focused during class

Todd Martindale

Age: 13

Background: Hyper, Distracted, Passive

Todd is an 8th grader at Elefante Middle School. Before the pandemic, Todd was an average student, but quarantine impacted his studies negatively. His teachers require him to keep his webcam on during class, but even so Todd has difficulty focusing in class due to distractions like social media, games, video calling friends, and eating. He listens when his teachers present but engages as sparingly as he can with the class.

Motivations:

• Complete his classes with minimal effort
• Find enjoyment outside school

Frustrations:

• Focusing for the class content and the teacher is much harder than before, but since there’s no easy way to do so, Todd doesn’t do anything
• His grades are slipping

Competitive Audit

Prior to working on prototypes, we also wanted to analyze the features that are/aren’t offered by other virtual classroom platforms.

Despite the fact that most online virtual classroom platforms have these features for teaching signaling/messaging, users are still dissatisfied with the effectiveness and quality of these communication features. Knowing that other platforms already have these features, but still fail to satisfy their respective user bases with the current functionality, gives us an idea of where we want to head with our UX redesign/improvement. We now know where each platform is lacking with respect to interactivity between meeting participants and can suggest measures or improvements to counteract the concerns of the users.

UX Flows

Student Flow

Our UX Flow’s entry point is when the user opens Zoom, which we chose to be the entry point because the user cannot access the participants functionality without opening Zoom itself. The first flow represents the student’s UX. When the student is confused, they are given resources (such as Select Mood, View Mood Board) that can influence the teacher and subsequently their learning. Thus, they have a simple but effective way to increase interactivity and engagement with the professor and class material.

Teacher Flow

The second flow represents the teacher’s UX, which is simpler than the student’s. As they are teaching, if they receive a notification that half the class is confused, they have the opportunity to change their teaching behavior to increase their class’s effectiveness and improve student comprehension. Our main flow is for our primary persona (students) and our second flow is for our secondary persona (teachers). Our UX flows were not inspired by any apps.

UI Sketches

We learned that we have an approximate idea of what we’re going for, but there are many small details worth exploring that make it hard to decide on a final design. The sketches represent each window mentioned in our UX flow: Participants window, Mood Board, and Pop Up Notification, and take into account whether Zoom is full screen or windowed, and if any other part of Zoom such as the chat function is open. Our sketches were not inspired by apps in our competitive audit.

Low Fidelity Prototypes

We created two different versions of low fidelity prototypes to see which one users liked more during user testing.

Prototype A:

Prototype B:

We designed this prototype to build on the previous prototype but by representing the UI and flow in a different direction. All the mood options are listed in a popup on the lower bar instead, and the user can view the Mood Chart by clicking “Statistics”. It takes a completely different approach from the UI options we sketched before by making a simplified short-cut version to represent our student flow.

User Testing

We conducted user testing with three K-12 students by allowing them to interact with both of our prototypes and perform certain tasks.

The three user testing interviews we conducted yielded very similar results. In general, when our users needed help or wanted to ask a question in class, they would utilize the chat functionality of Zoom to ask their instructor or they would have to ask them outside of class. Most of them agreed that this was not very effective as the instructor sometimes has too many questions coming in to get to everyone or the instructor was simply just not accepting questions. When it came to evaluating our prototypes, all of the users agreed that Prototype B was the more intuitive and accessible one. Some of the problems they mentioned with Prototype A was that most students just don’t open the Participants’ Window during class and that the design lacked consistency in how it presented many of its UI components. What they liked about Prototype B was the accessibility and the simplicity of the UI layout. They also liked that it signaled to the teacher that they were confused with the question mark. Some of the suggestions they had for improving Prototype B were to add the option for moods to be private, to add a quantity or percentage to the mood chart, and to change the appearance of the visual components.

High Fidelity Prototype

After extensive user testing and a wide amount of peer feedback, we created our finalized high-fidelity prototype.

Before and After Stories

We actually had a couple versions of the high fidelity prototype before we arrived here. Through peer feedback and user feedback, we were able to refine our prototype and address problems raised as a result from testing.

Note: Left=Before and Right=After. We received feedback about changing the sample names to actual names, so we updated that by replacing Person 1, Person 2, etc. with made up names. Since the center screen was too distracting, we replaced it with Google’s search page, which is a simpler example of what someone may be sharing their screen of. Lastly, the mood icons’ designs were inconsistent, so we changed them to all include circular backgrounds and white icons.

We decided “happy” was less useful for students and teachers, and “raise hand” would be used more in the mood board. Many people did not know what “statistics” meant either, so we changed its label to “view statistics”.

Note: Left=Before and Right=After. We came to the conclusion that having the statistics chart appear in the Participants window is not convenient given that users do not use this window that often to begin with. Instead, we opted to make the statistics chart appear as a pop-out window.