(updated September 2018)
In an earlier blog post, we talked about using the Visualization Toolkit (VTK) and the Vive to explore your data in Virtual Reality (VR). In this post, we are going to talk about the Virtual Reality capabilities in ParaView with support for the Oculus Rift, HTC Vive and Windows Enhanced Reality headsets.
Before we dig into how to get your data into VR, we really want to thank the National Institutes of Health, who sponsored the work under the grant NIHR01EB014955 “Accelerating Community-Driven Medical Innovation with VTK,” our commercial customers, and our collaborators at the Oak Ridge Leadership Computing Facility (OLCF).
Getting Some Hardware
ParaView will work with any headset supported by the OpenVR standard. This includes the Oculus Rift, HTC Vive, and Windows Enhanced Reality headsets. Currently, ParaView’s support for the HTC Vive/Vive Pro is better, so if you are undecided, have your purchasing department buy that first. You can purchase the Vive from Amazon or directly from HTC; as of this writing, it runs about $500. If you want to go the Oculus Rift route, you can buy one directly from Oculus or from some retailers such as Best Buy.
A good VR experience requires very high frame rates, typically 90 Hz per eye. This means ParaView needs to render very quickly. Having a high-end CPU/GPU is critical. Both the Rift and Vive suggest at least a GTX 970 class GPU, but the higher, the better. Both vendors sell or recommend systems that are Vive-ready PCs or Oculus-ready PCs. Likewise, both vendors have tools to check if your current system is VR ready for the Vive or Oculus.
Once you have purchased a headset, make sure that it is installed per the manufacturer’s instructions and that it works with their demo software. For all headsets you also need to download and install SteamVR as we use the OpenVR API as the interface.
VR support in ParaView
In the past we required specific VR builds for ParaView but as of mid 2018 ParaView comes with support for VR built in. It is in source tree and is included in the windows binaries that you can obtain from the download page. It is designed to run under a 64-bit version of Microsoft Windows such as Windows 10. The OpenVR support is a plugin in ParaView. To use it you must first load it from the Tools -> Manage Plugins menu option. Select OpenVR and load it. This will add a OpenVR specific panel to your user interface.
Before running this version of ParaView, make sure Steam VR is running. Once installed, you can then run ParaView by double clicking on the executable. ParaView will start as usual with a window on the computer screen. Using the regular mouse/keyboard, you can load or create your visualization as you normally would in ParaView. ParaView also has a full tutorial you can download.
Once you have your visualization up in ParaView, you can send it to VR by hitting the “Send to OpenVR” button in the OpenVR plugin panel (note the image below is out of date).
You can then interact with the data using the two touch controllers. Pulling the trigger will grab an object, allowing you to hold it and move it around. If you hold both grip buttons down at the same time, you can scale and translate the virtual world. Pulling your hands apart will make it larger while moving them closer together makes it smaller. Moving your hands in the same direction translates the world in that direction. Pressing down on the touch-pad or joystick will start flying in the direction the controller is pointing.
When you are done in VR, press the menu button on a controller (application menu for Vive, A button for Oculus) to return to ParaView. Now you can make more changes to your data in ParaView, and when you are done, hit the “Send to OpenVR” button to return to VR. This way, you can alternate between exploring your data on the screen and in VR.
Note that animations may not work in VR and are still in development. Many annotations will not show up. Finally, it does not support remote rendering of any kind due to the high frame rates required.
Recent additions to VTK that make this all work
Under the hood, ParaView is using the new OpenVR rendering module in VTK. This module can be found in the Rendering sub-directory and can be turned on when compiling VTK using CMake options. When VTK is built with these modules, it includes new subclasses of vtkOpenGLRenderWindow, vtkOpenGLRenderer, vtkOpenGLCamera, and vtkRenderWindowInteractor to support rendering and interacting with the OpenVR. In addition, there are some new classes in RenderingCore designed to help with picking and interaction, when you have devices that support 3D positions and orientations such as the Vive and Oculus controllers. vtkRenderWindowInteractor3D extends many of the methods in vtkRenderWindowInteractor to include a Z coordinate and allow for event coordinates to be in world coordinates in additional to window coordinates. vtkPropPicker3D is designed to perform a pick using a 3D location as opposed to the more common X, Y screen location. Finally, vtkInteractorStyle3D provides the basic interaction style that the Vive controllers use to select and move objects or translate and scale the world.
If you have built VTK with OpenVR support, then creating an immersive visualization follows the same model as existing VTK visualizations. A simple cone example is shown below.
int main(int argc, char *argv)
Since this original post support for 3D widgets in VR has been added as well as support for volume rendering and numerous interactions in VR. In the future we plan to continue developing these capabilities including collaboration. ParaView has a number of 2D annotations and rendering options that would fit well on a popup sheet in VR.
If you have an application or technology area that could benefit from visualizations in VR, feel free to contact us. There are a lot of neat opportunities in this space, and we are excited to see where it goes.