Perspective Correction changes the way we see video meetings

With Jan Tore Korneliussen, Technical Product Manager for Imaging

Another piece of the puzzle

Image quality has always been central to Huddly, and tackling distortions has been one of our foundational goals. Since Huddly's early days, we've used dewarping algorithms to correct wide-angle lens distortions, eliminating the fisheye effect. But perspective distortions? That turned out to be a whole new level of challenge.

Perspective distortions happen when we project a 3D world onto a 2D canvas, causing images to look stretched or tilted. The further from the center of the screen you sit, the more pronounced this effect gets.

"With more flexible mounting, Perspective Correction improves awkward angles and provides an eye-level view as if you're speaking face to face."

– Xiyu Hu, User Experience Designer

In photography, these distortions can be fixed in post-editing with tools like Photoshop, and live TV crews can rotate cameras to adjust the perspective. However, video conferencing has not addressed this until now.

In a recent article about the benefits of Edge AI, we discussed how we've all settled for a video meeting experience that simply isn’t good enough. Distortions are a part of that subpar experience.

The making of a Huddly feature

We began looking for potential solutions during one of our innovation weeks. These are dedicated times where our engineers are given time to explore their ideas and experiment with new technologies.

Our question was simple: Can we digitally rotate the camera to adjust the perspective and correct these distortions in real-time? One challenge was ensuring important image cues, like people and interior details, stayed aligned with gravity during rotation.

To solve this, we used an accelerometer, or gravity sensor, and reliably detect which way is up. This keeps the image naturally oriented to the real-world perspective.

From there, we identified three core requirements for a robust feature: eliminate distortions of people, align with gravity, and integrate seamlessly with our AI framing. Through many iterations, we developed Perspective Correction, a solution that leverages hardware, software, and AI.

Once you see it in action, you won’t want to go back to the old view.

"For us, it’s not about correction for its own sake—sometimes, a tilt is okay. We prioritize how people look over perfect spatial accuracy."

– Christina Karam, Senior Imaging Engineer

What does Perspective Correction do?

First off, it removes that warped effect that can make the edges look stretched or out of proportion. With Perspective Correction, everyone is represented naturally and no one’s at an advantage or disadvantage just because of where they are in the space.

Second, this feature corrects placement and tilt issues that typically come from certain camera positions. Imagine a camera positioned high up, angled downward. Without correction, it can make people look as if they’re toppling over and lines as if they're tilting. Perspective Correction digitally rotates the angle to adjust the perspective, bringing stability that aligns with our natural sense of gravity.

Finally, Perspective Correction operates in real-time with our AI framing experiences, such as Huddly Director and Group Framing. It adapts to room dynamics to give a natural representation of people and spaces.

Innovation doesn’t happen overnight

Having an idea is the easiest part of the innovation process. The real work is taking it to completion, even when things get tough. We make innovation and iteration a natural part of our work, cycling through trial and error, user feedback, and adjustments. That is how we turn vague concepts into solid technology that brings real value to our users.