Memory compression in ITK-SNAP

ITK-SNAP is a software application used to segment structures in 3D medical images. It is the product of a decade-long collaboration between Paul Yushkevich, Ph.D., of the Penn Image Computing and Science Laboratory (PICSL) at the University of Pennsylvania, and Guido Gerig, Ph.D., of the Scientific Computing and Imaging Institute (SCI) at the University of Utah, whose vision was to create a tool that would be dedicated to a specific function, segmentation, and would be easy to use and learn. ITK-SNAP is free, open-source, and multi-platform.

Internally, ITK-SNAP keeps both image grey-values and segmentation labels as 16-bit integers. It also keeps an undo image, so normally 6 bytes are allocated for each image voxel. The segmentation layer usually contains small number of labels, and they are grouped together into large chunks, which makes it a perfect target for run-length encoding (RLE) to save working memory (RAM). The savings are significant for large images.

ITK-SNAP Version Image (Size MVox) Number of labels Private Working Set (MB) Commit Size (MB) PWS difference Commit difference
3.x nightly  None (0) 0 75 154 100% 100%
3.x nightly vb-seg (116) 9 626 731 100% 100%
3.x nightly wb-seg (390) 58672 2912 3058 100% 100%
dev_3.6 None (0) 0 80 165 +6% +7%
dev_3.6 vb-seg (116) 9 403 508 -36% -31%
dev_3.6 wb-seg (390) 58672 2032 2267 -30% -26%

The above table depicts memory savings of RLE implementation. 6% higher memory consumption without any image loaded probably comes from differences in compiler used (3.x nightly was downloaded, dev_3.6 locally compiled). The private working set is physical memory used, commit size is total allocated memory of the process. The decrease is slightly less than a third. Considering that about a third of the memory is dedicated to segmentation image, it means that RLE is very effective for the two typical test images used. In fact, the RLE compression ratio is 44x and 20x for the two test images used!

A run-length encoded image type (RLEImage) was implemented, designed to fit into Insight ToolKit (ITK), along with accompanying iterators and a region of interest image filter. Region of interest image filter has three scpecialized variants, so it can be used both to extract a subregion from an RLEImage, as well as convert between itk::Image and RLEImage (optionally with a subregion operation).

As image slicing is executed whenever mouse is scrolled or clicked, special care has been taken to keep it fast. To this end, slicing filter has a multithreaded implementation. For slicing planes which are parallel to RLE axis it is even faster than slicing without memory compression!

Durations in milliseconds Dimensions itk::LabelMap naiveRLE SNAP+RLE SNAP-noRLE pureITK
Image Labels X Y Z X Y Z X Y Z X Y Z X Y Z X Y Z
MRIcrop-seg.gipl.gz 9 78 110 64 5.1 3.9 4.1 3.0 2.7 2.7 1.3 0.2 0.3 0.5 0.4 0.5 2.0 0.3 0.3
brainParc.mha 114 256 256 256 51.0 39.2 39.0 33.3 28.3 28.2 11.7 1.1 1.0 5.2 3.6 3.6 10.3 0.5 0.4
vb-seg.mha 9 640 633 299 105.9 71.5 94.8 68.0 61.1 63.1 30.0 2.9 5.6 22.5 10.4 21.8 26.3 0.6 0.7
wb-seg.nrrd 58672 512 512 1559 993.6 912.1 874.7 472.1 392.6 386.6 154.2 15.4 8.1 73.9 43.4 14.1 79.1 1.4 0.6

The above table shows average slicing duration (in milliseconds) of several different slicing implementations. ITK-SNAP's pre-existing slicing code without memory compression is somewhat slower than pure ITK implementation because it does image flipping on the fly, and it is single-threaded too. The case of slicing perpendicular to X (RLE axis) is slower than non-compressed version, but only about twice slower. Slicing along other axes is faster, in same cases more than twice faster, so dual-core multi-threading alone cannot explain it. My conclusion is that taking advantage of RLE structure is reducing number of memory reads, which is the expensive operation in today's computers.

The code is currently commited to dev_3.6 git branch of ITK-SNAP. Once its stability has been proven, it could be integrated it into Insight ToolKit (ITK). This will enable other applications to take advantage of RLE compressed images.

Questions or comments are always welcome!