Taking ParaView into Virtual Reality

In an earlier blog post, we talked about using the Visualization Toolkit (VTK) and the Vive to explore your data in Virtual Reality (VR).  In this post, we are going to talk about an experimental version of ParaView you can download that adds VR support for both the Oculus Rift and HTC Vive headsets.

Before we dig into how to get your data into VR, we really want to thank the National Institutes of Health, who sponsored the work under the grant NIHR01EB014955 “Accelerating Community-Driven Medical Innovation with VTK,” and our collaborators at the Oak Ridge Leadership Computing Facility (OLCF).

Getting Some Hardware

The VR version of ParaView will work with both Oculus and the HTC Vive headsets. Currently, ParaView’s support for the HTC Vive is better, so if you are undecided, have your purchasing department buy that first. You can purchase the Vive from Amazon or directly from HTC; as of this writing, it runs about $800. If you want to go the Oculus Rift route, you can buy one directly from Oculus or from some retailers such as Best Buy (currently for $600).

A good VR experience requires very high frame rates, typically 90 Hz per eye. This means ParaView needs to render very quickly. Having a high-end CPU/GPU is critical. Both the Rift and Vive suggest at least a GTX 970 class GPU, but the higher, the better. Both vendors sell or recommend systems that are Vive-ready PCs or Oculus-ready PCs. Likewise, both vendors have tools to check if your current system is VR ready for the Vive or Oculus.

Once you have purchased a headset, make sure that it is installed per the manufacturer’s instructions and that it works with their demo software. For the Oculus, you need to make sure you authorize running “Unknown Sources” in the Settings/General menu as shown below.

 

Oculus

 

The VR version of ParaView

First, download the experimental, VR version of ParaView from the ParaView download page. It can be found under Releases with the Version of ParaView set to v5.2 and the Type of Download set to Experimental Demos. It is designed to run under a 64-bit version of Microsoft Windows such as Windows 10 64-bit. You can also download it directly. Once downloaded, unzip the file and place it on your system. The zip file includes the ParaView executable along with the required DLLs to run it.

Before running this version of ParaView, make sure Oculus Home for the Rift or Steam VR for the Vive is running. Once unzipped, you can then run ParaView by double clicking on the executable. ParaView will start as usual with a window on the computer screen. Using the regular mouse/keyboard, you can load or create your visualization as you normally would in ParaView. If you are new to ParaView, you can get a quick start by loading the sample ParaView State file SampleVRTubes.pvsm, which is included in the zip file. ParaView also has a full tutorial you can download.

Once you have your visualization up in ParaView, you can send it to VR by hitting the “Send to Oculus” button for Oculus Rift or the “Send to OpenVR” button for the HTC Vive. These buttons are located at the bottom of the Properties Panel as shown in the lower-left corner of the below image.

PVVR2

 

PVVR2SendWhile in VR, you should not interact with the ParaView window. With the HTC Vive, you can interact with the data using the two touch controllers. Pulling the trigger will grab an object, allowing you to hold it and move it around. If you hold both triggers down at the same time, you can scale and translate the virtual world. Pulling your hands apart will make it larger while moving them closer together makes it smaller. Moving your hands in the same direction translates the world in that direction. Pressing down on the touch-pad will start flying you in the direction the touch-pad is pointing.

When you are done in VR, press the menu button on a touch controller on the Vive to return to ParaView or the space bar on the keyboard for the Oculus Rift. Now you can make more changes to your data in ParaView, and when you are done, hit the “Send to OpenVR/Oculus” button to return to VR. This way, you can alternate between exploring your data on the screen and in VR.

This experimental version should handle rendering polygonal surfaces and molecular models. It currently does not support volume rendering, although the underlying support in the VTK layer does. We suspect it will not work with animations either, so be warned. Many annotations will not show up. Finally, it does not support remote rendering of any kind due to the high frame rates required.

Recent additions to VTK that make this all work

Under the hood, ParaView is using the new Oculus and OpenVR rendering modules in VTK. These two modules can be found in the Rendering sub-directory and can be turned on when compiling VTK using CMake options. When VTK is built with these modules, it includes new subclasses of vtkOpenGLRenderWindow, vtkOpenGLRenderer, vtkOpenGLCamera, and vtkRenderWindowInteractor to support rendering and interacting with the Oculus or Vive, respectively. In addition, there are some new classes in RenderingCore designed to help with picking and interaction, when you have devices that support 3D positions and orientations such as the Vive and Oculus controllers. vtkRenderWindowInteractor3D extends many of the methods in vtkRenderWindowInteractor to include a Z coordinate and allow for event coordinates to be in world coordinates in additional to window coordinates. vtkPropPicker3D is designed to perform a pick using a 3D location as opposed to the more common X, Y screen location. Finally, vtkInteractorStyle3D provides the basic interaction style that the Vive controllers use to select and move objects or translate and scale the world.

If you have built VTK with either Oculus or OpenVR support, then creating an immersive visualization follows the same model as existing VTK visualizations. A simple cone example is shown below.

#include "vtkActor.h"
#include "vtkConeSource.h"
#include "vtkNew.h"
#include "vtkOpenVRCamera.h"
#include "vtkOpenVRRenderer.h"
#include "vtkOpenVRRenderWindow.h"
#include "vtkOpenVRRenderWindowInteractor.h"
#include "vtkPolyDataMapper.h"

int main(int argc, char *argv[])
{
 vtkNew<vtkActor> actor;
 vtkNew<vtkOpenVRRenderer> renderer;
 vtkNew<vtkOpenVRRenderWindow> renderWindow;
 vtkNew<vtkOpenVRRenderWindowInteractor> iren;
 vtkNew<vtkOpenVRCamera> cam;

 renderWindow->AddRenderer(renderer.Get());
 renderer->AddActor(actor.Get());
 iren->SetRenderWindow(renderWindow.Get());
 renderer->SetActiveCamera(cam.Get());

 vtkNew<vtkConeSource> cone;
 vtkNew<vtkPolyDataMapper> mapper;
 mapper->SetInputConnection(cone->GetOutputPort());
 actor->SetMapper(mapper.Get());

 renderer->ResetCamera();
 renderWindow->Render();
 iren->Start();
 return 0;
}

Future Directions

There are a number of areas we are looking to work on in the future. Oculus should be releasing their touch controllers soon, at which point we hope to add the same interaction options to the Oculus Rift that we have currently for the HTC Vive. We want to enable volume rendering and animations from ParaView to be exported to VR. ParaView has a number of 2D annotations and rendering options that would fit well on a popup sheet in VR. Lastly, 3D controllers with their real-time position and orientation open up a wide range of possibilities in terms of interactive widgets to explore and steer the visualization.

If you have an application or technology area that could benefit from visualizations in VR, feel free to contact us. There are a lot of neat opportunities in this space, and we are excited to see where it goes.

 

11 Responses to Taking ParaView into Virtual Reality

  1. Will Schroeder says:

    +100 well done

  2. Tristan Wright says:

    With the Vive can you move around in the virtual space?

    • Ken Martin says:

      Most definitely. Your head position and orientation are tracked for starters. But beyond that, using the two Vive handheld controllers you can scale and translate the world as well as fly around.

  3. It works really well! Would be great if we can add animations

  4. Axel Gerstenberger says:

    I do not have either headset, but, conceptually, would the same work with a VR Cave that has tracking? Normally, we use ParaViews VR Plugin, but this “Send to Open VR” looks like it could work with caves as well? This may allow to prepare complex scenes, while VR users look at the previous scene until you press “Send” again. Just a thought…

    +1 for animations

  5. Guillaume Jacquenot says:

    Very interesting job. It opens a wide bunch of possibilities. Is there any plan to release a full version of ParaView for Windows (i.e. with Python scripting functionnalities)? And also a version with animations?

    • Ken Martin says:

      I am working on a new paraview release with improved VR support and some additional features. I’ll try to make sure I build it with python scripting included. I do hope to get the changes merged into paraview master at some point as well.

  6. Benjamin says:

    Note that If you are running Paraview with the Vive, it will only work if the SteamVR Room Setup is configured in “Room-Scale” mode. At least for me “Standing only” mode did not work.

  7. YueHan Zhu says:

    hello! I am working on showing the processing result of openfoam to VR, and the biggest issue is exporting the 3D animation. Did the file which exported by “foamToVTK” contains all the frames from openfoam? thanks!

Questions or comments are always welcome!