This blog summarizes a case study. To read the full case study, click here.

As schools have shifted to remote learning, Google Classroom has become nearly ubiquitous in New York City’s schools. Google Classroom captures data about student engagement, in a way that isn’t always user friendly or actionable for educators. New Visions has worked with partner schools to turn data into actionable insight.

New Visions and our partner schools define “student engagement” in Google Classroom simply: a student “engages” on any day in which she submits an assignment in Google Classroom. This is a simple definition of engagement and a lower bar than any of our ambitions for remote instruction, but it follows the approach that New Visions has used in all our data work: to rapidly provide visibility into key student and educator behaviors and decisions that influence student success, then use that information to refine measures and improve practice in partnership with educators working in schools every day.

The Khalil Gibran International Academy (KGIA) is a small high school in downtown Brooklyn. Carl Manalo became principal of KGIA in 2019. When New Visions asked for partners, Principal Manalo did what he always does: he raised his hand.

With access to Google Classroom data, Principal Manalo and his team took action: they identified students with low engagement rates, reached out to understand why they weren’t engaging and to reinforce the importance of completing work. Over a three-day period in April, the school saw improvement in engagement rates across all student cohorts, with school-wide engagement rates rising from 55 percent to 62 percent. Seeing an uptick of several percent is huge and has gotten more staff members invested in using these data, Principal Manalo reflected. Staff members feel more connected to students and able to support their needs, even if they are seeing students on computer screens rather than in the hallway.

The path that KGIA took to improve engagement in a remote learning world won’t work for every school, but that’s not the point. There are specific lessons to draw from the way they worked, that pop up again and again in the continuous improvement work New Visions does with schools. Briefly:

 

The predicate for all of this work is having some meaningful data available. Turning in assignments on a daily basis is not perfect data, but it’s much better than nothing. Once those data are visible, it’s possible to use them to guide decisions and action.
 

 “Student engagement” during remote learning is a novel metric for most schools, and it was for KGIA too. That didn’t stop them from using data they could see to set a goal. They looked at students who they had a reasonable suspicion were engaging in ways they wanted, and said, let’s work to get all our students there.

 

KGIA set a reasonable and achievable improvement goal: moving several percentage points in daily engagement, over a short period of time. It’s big enough to be measurable, and small enough to be achievable and testable quickly.

 

Before the brand-new Data Portal engagement features were developed (an MVP, in technology development parlance), Principal Manalo took screen shots of the engagement rate each day in order to track progress. KGIA didn’t let the absence of some features stop them.

 

Tracking data on a daily basis let KGIA make reasonable inferences about whether their actions were making a difference. When they saw improvement in the metric that mattered to them, they took a moment to celebrate hard work and improvement! That’s always important, but it’s critical during these challenging days. Mission accomplished? Not forever, but for now!

 

Principal Manalo incisively observed that once teachers could see the impact of their actions on something that mattered to them—student engagement—they wanted to improve even more. The data discussions that started in the Senior Success Team are now part of department team meetings, and all departments in the school are involved.

 

As KGIA educators got more invested and fluent in their data, they realized that they were using Google Classroom differently. Some were treating “Do Nows” as assignments in Google Classroom. When students didn’t complete them, their assignment completion rates looked artificially low. Do Nows are important, but for KGIA they aren’t actually assignments, and seeing the data helped them realize they were treating them as such. This realization let them norm on practice, while at the same time improving the quality of their data.

 

Real, lasting improvement requires lots of small experiments, accumulating into sizable gains for students and durable changes in practice at KGIA (and hopefully, over time, in other schools wrestling with similar challenges who can learn from the KGIA example).

The team at KGIA is working to change how their school works, in creative and disciplined and student-centered ways. The changes they’re making are critical now, but they’ll also change how the school works moving forward. Principal Manalo predicts that many of the teachers that have started using Google Classroom during this crisis will continue to do so once the crisis is past, and teachers that were using it before will use it in different ways. Gaining visibility into which classes are engaging students allows the school team to identify and learn from effective practices happening inside their school, using data for improvement rather than accountability.