BETA Release

Catalyst VR

A VR plug-in for Catalyst Edit that enables content creators to edit their 360-degree footage within a virtual environment, making it easier to preview their 360-degree edits real-time. 

Company

Sony

Role

Lead Product Designer

Date

2017 - 2018

Project Background

As 360-degree video capture became increasingly popular amongst content creators, the tools available in the post-process workflow had simply not kept up. Our goal was to adapt our existing desktop-based non-linear video editing software to better suit the 360-degree video editing workflow. To do this, we flew around the country speaking to content creators in order to better understand their user needs and current pain points. One of the primary pain points we identified was the ability to review footage using a VR headset. At the time, editors would edit their footage using a fairly distorted equirectangular format on a traditional computer monitor. They would then export their video to a head-mounted display for review. This meant there was no way to know for certain whether the 360-video actually looked good until exporting the video, and then reviewing it in an actual VR headset. 360-degree video generally required multiple tweaks to orientation and projection mapping, and cutting between two scenes required precise alignment of the horizons, and points of interest, to ensure a seamless transition.

My Role

As the lead designer of Catalyst Edit VR, I worked closely with our engineering teams in India and Japan to create a VR experience that enabled creators to easily edit their content using HTC Vive controllers and an HTC Vive headset. As VR applications require 3D assets and a very different design approach from tradition 2D UI, I learned to design 3D models using Blender and created a new design process that allowed me to communicate my design decisions our development team.

Solution

My team and I developed an application that allowed video editors to edit their footage in virtual reality. We wanted to minimize the task switching costs of taking the VR headset on and off, and bring the editing and review experiences together into a single workspace. To do this, we projected their 360-degree footage into their virtual world - essentially making the footage their own virtual editing environment. We then modified our Catalyst Edit timeline to be better optimized for VR and mapped numerous editing and playback controls to the HTC Vive controllers. Using the controllers, users could playback and modify their footage, all within the Vive headset. This provided a real-time playback and editing experience, making it extremely easy to line up video cuts and re-orient footage.

This prototype and research would lead to Sony filing a patent on behalf of myself, engineering manager Tanaka-San, and engineering lead Ihara-San:

Patent: Image processing apparatus and method, and program
 

Prototype