For the past few years, Virtual Reality have become a rapidly growing technology because of its awesome experience and has become a vision of video gaming’s promising future. But that leads to some very unconventional obstacles too. One of the major challenges is the user interface design and the user experience. We interact with digital interfaces on our phones, our tablets, our televisions, our cars, our clothing, our refrigerators, etc. And with each new input device comes the necessity to rethink how we design the interfaces they require. Same is the case with the virtual reality.

Valve Software, while experimenting with virtual reality, has figured out two factors which have an immense influence on virtual reality UI design: occlusion and convergence. Occlusion means how to manage the UI layers depth i.e the Z order. i.e. what is in front of what? Convergence means the point to which the focus of our eyes meet, also called gaze. Drawing the interface directly onto the screen, as would normally happen, causes conflicts. Both eyes would essentially be looking in parallel lines of focus which is called gaze, a cue which tells your brain that the thing you’re looking at is far away. In this case we cannot render the UI on back buffer. Therefore, the interface needs to exist somewhere within the virtual 3D space created by the Oculus. It also needs to exist near the center of your viewpoint. No more corner anchored designs. No one wants to move their eyes way into their periphery to see a menu item. Essentially it would be better if Oculus headset has an eye tracking feature (which will be available in the future). However, a Germany based company SensoMotoric Instruments GmbH (SMI) developed a pretty cool eye-tracking prototype integrated with Oculus Rift. Other hacks are also available by some indie people.

We can make a 2D interface and simply make it a flat, planar object to insert in front of the users face. Or we could take that 2D interface and wrap it around the users head as if projected onto an invisible sphere or like an Iron Man style helmet visor. This also brings up issues with stereoscopy. Placing the user interface too close to the user’s face could cause retinal disorder, a condition where your eyes see two totally different, overlapping views causing confusion as to which eye sees what. Although it makes sense to attempt to mimic a real life interface, for example being in a virtual music studio surrounded by the interface as a producer would be surrounded by his audio equipment. As its virtual reality, so it will be better if the the user experience could be 3D immersive. i.e. manipulation of 3D virtual environment objects in multi-user 3D virtual environments to make them interactive.

For this purpose, I have found probably the best UI solution dedicated for making immersive UI called Scaleform. Its a middleware UI solution that gives leverage to developers to use Flash based UI in leading game engines like Unity, UDK, CryEngine etc. Comes with pretty handy SDK for Unreal and Unity, Scaleform provides a design-driven workflow for creating robust, optimized and cool graphics and UI elements. All of the documentation is well elaborated and easy to understand. In my future posts, I will share my findings about the ins and outs of Scaleform regarding UI development.