[VTK 6] New feature : Picking Manager

 VTK provides a variety of classes to perform actor, point, cell, and world point picking:

  • vtkCellPicker
  • vtkPropPicker
  • vtkWorldPointPicker

Depending on the type of picker used, either software-based geometric intersection or hardware picking is applied. While all the pickers work appropriately, to date, there is no communication between pickers.

This shortage of communication leads to problems with the interaction, in particular within the VTK widgets:

  • For a given position, multiple props can be picked by different pickers. This is typically an issue when hovering VTK widgets. At a given mouse point,  more than 1 VTK widget can be hovered (there is no grabFocus() to break the callbacks) at the same time. See Figures 1a/1b for an illustration. 
  • A prop, can be picked despite being further away from the camera than other pickable props. Depending on the order of insertion of mouse press event observations, a VTK widget – first in the callback list – can grab the focus on a mouse click preventing the remaining VTK widgets from being picked. See figures 4a/4b for an example.

Figure 1a

Figure 1b

Figure 1a – No Picking Manager (or disabled)
Both vtkBoxWidget2 and vtkBalloonWidget (associated with the cone) are picked/highlighted on mouse hover 
Figure 1b – Picking Manager enabled
Only the vtkBoxWidget2 handler is picked/highlighted. 

In both cases, the root cause of the problem comes from a lack of “communication” between pickers. Based on its awareness (vtkAbstractPicker::PickList), a picker returns a picked prop that might not be the best picked prop in the scene.

This is the reason why a picking manager (vtkPickingManager) is introduced into VTK. Its role is to coordinate picking across all widgets simultaneously by implementing a more natural interaction based on the concept of “What you see is what you get”. In practice, the Picking Manager maintains a collection of registered pickers and every time a Pick() is issued by a VTK widget, the manager evaluates every registered pickers and selects the “best” picker, the one that returns a prop the closest from the camera. It then returns the widget/representation of the picker that was chosen or the picker itself.

Implementation:

Much effort has been made in order to keep the integration within VTK as minimal as possible. Nonetheless, in order to ease the use of the picking manager and to keep as minimal the code required to activate the management, picking manager awareness has been added to VTK widgets and representations.

Note: vtkInteractorObserver is the base class of all the VTK widgets (e.g. vtkHandleWidgetvtkImplicitPlaneWidget) and the vtkWidgetRepresentation is the representation for different types of widgets (e.g. vtkHandleRepresentationvtkImplicitPlaneRepresentation).

PickingManager Process

 Figure 2 – High Level implementation schema

 In Practice:

Every time a vtkWidget or a vtkWidgetRepresentation is instantiated, the widget or representation automatically registers its picker(s) and starts to be managed. For fine-tuning, it is possible to manually interact with the management in two ways:

  • For each individual widgets, vtkInteractorObserver::PickingManaged or vtkWidgetRepresentation::PickingManaged can be turned on/off to  determine whether it uses the manager or not. 
  • It is also possible to directly disable the manager itself with the methods vtkPickingManager::EnableOn()/EnableOff()/SetEnable(bool). If disabled, the VTK widgets/representations will behave as before.

PickingManager Diagram

 Figure 3 – Picking manager workflow

Here is an example to activate picking manager for all the widgets in the scene:

vtkNew<vtkRenderer> renderer;
vtkNew<vtkRenderWindow> renWin;
vtkNew<vtkRenderWindowInteractor> iren;

vtkNew<vtkSeedWidget> seedWidget;
seedWidget->SetRepresentation(seedRepresentation.GetPointer());
seedWidget->SetInteractor(iren.GetPointer());
seedWidget->EnabledOn();
for (…)
  {
  vtkHandleWidget* newHandle = seedWidget->CreateNewHandle();
  newHandle->EnabledOn();
  }
iren->GetPickingManager()->EnabledOn();
iren->Start();
return EXIT_SUCCESS;

 

Results & Performances:

We have good results and several demos are available highlighting the Picking Manager in different use cases. The images below compare scenes when the Picking Manager is disabled versus enabled.

Figure 4a – Picking Manager off/disabled
Two spheres are under the mouse cursor. Here the firstly created sphere is picked/highlighted (grabFocus()) but it is not the “closest” sphere
Figure 4b – Picking Manager on/enabled
Despite being last in the mouse press callback list, the sphere representation closest to camera is correctly picked/highlighted

A video showing the picking manager in action is available here.

In terms of performance, we are very close to matching Picking Manager-free performance, with an average speed drop of only 6%within a scene involving 64 seed widgets:

The graphs represent the time in seconds taken by an action versus the action ID; blue represents interaction without activating the Picking Manager and green represents interaction with it activated. On the left side is the immediate time taken for each action; on the right is the cumulative time.

Future Direction:

We are investigating how to allow different strategies for the picking selection based on some bitfield flags.

We are working on the processes involving the vtkMultiblockDataSet and, more particularly, on a vtkCompositeActor, and will then focus on making the use and integration of the Manager clean and easy.

We also plan to optimize the Manager use during interaction to avoid methods such as vtkRenderWindowInteractor::GrabFocus(), which we could follow with cutting off the research space to use the bounding boxes of the registered objects to only focus on a few pickers.

The Picking Manager can also be speed up  by parallelizing non hardware pickers.

Conclusion:

We have provided a proof of concept of a Picking Manager for VTK 6.0 with convincing results. The integration within VTK stays limited and performance is not dramatically affected during management. This feature is important in the development of VTK and fixes major interaction issues seen in situations using the composite actor or other interactions such as with the vtkButtons.

The Picking Manager will not be active by default as it slightly reduces performance when interacting with the scene.

Acknowledgements:

The commit associated with the Picking Manager is available at http://review.source.kitware.com/#/c/6642/.

The development effort for the Picking Manager is part of the MSV project. For information, the MSV official website  has further details. You can also find the complete source code and follow the other features’ development on the git repository: https://github.com/MSV-Project.

5 Responses to [VTK 6] New feature : Picking Manager

  1. Stephen Aylward says:

    Outstanding work! Congratulations Michael!

  2. Will Schroeder says:

    These are great additions solving some very hard problems. Thanks Michael!

  3. Petar Petrov says:

    very interesting! never had the issue, since hardly having more than one prop at close proximity, but I guess the problem might manifest itself at some point. so having a VTK6 build, I can already enable the picking manager?

    another frustration I have regarding props picking is: how do you identify which domain object you have picked?
    I guess i ca use: virtual int IsA (const char *type)
    but what if i have a composite object and i click on only one part(visible) of it?

    So I ended up using: virtual void vtkProp::SetPickable(int)
    and set the int from enum type where i have all my domain objects clickable!

    i consider it a hack! any advice?

  4. Julien Finet says:

    Peter, I’m not sure to understand your problem.
    If you want to pick a composite dataset, you need to use a vtkHardwareSelector and get the vtkSelectionNode::COMPOSITE_INDEX() of the returned selection node (works only with vtkCompositePolyDataMapper2).
    Are you doing something else ?

  5. Petar Petrov says:

    Sorry, I should have elaborated further.
    When I say composite object, I meant a custom domain (a.k.a. business logic) object (not inheriting from vtkObject) but it has member two vtkActors of type vtkSpher, just an example. Now i would like to have a setTag(int tag) or setTag(const char* tag) where I specify my own integer/string.
    So that, once I have a reference to the picked object I know how it relates to my custom domain objects. So i click on one of the spheres is read getTag() and now I know I have clicked on an apple, for example.

    I used to work with the old software picker where I remember that a reference to vtkProp3D or vtkActor was given on the callback, only! So i used the visibility int to work around it, since any positive integer is regarded as TRUE in c++ 🙂

Questions or comments are always welcome!