Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and S-PCNNs in HSV space

Computational imaging plays an important role in medical treatment for providing more comprehensive medical images. This work proposes a new scheme to fuse computed tomography (CT), magnetic res- onance (MRI), and positron emission tomography (PET) images into a single image. A novel two-stage medical image fusion scheme, which is based on non-subsampled shearlet transform (NSST) and simpli- fied pulse coupled neural networks (S-PCNNs), is proposed in the hue-saturation-value (HSV) color space. Firstly, CT and MRI images are decomposed into a set of low and high frequency coefficients by NSST, PET images are transformed into the HSV color space, and then the V component of PET image in the HSV color space. Secondly, intersecting cortical models (ICMs) are utilized to extract the edges and outlines in a larger area from the high frequency coefficients, and S-PCNNs are employed to describe the subtly de- tailed information in a smaller area. Thirdly, different fusion rules are designed to fuse the corresponding low and high frequency coefficients. At last, the fused medical image is obtained by the inverse HSV and NSST transformation, successively. The experimental results show that the proposed scheme is effective, and it can fuse more information into the final images than conventional methods.