Efficient Utilization of Image Fusion and Interpolation for Medical Image Diagnosis applications

— This paper presents a framework for medical image diagnosis of brain tumors. This framework comprises image fusion, image interpolation and image segmentation. The objective of the fusion process is to integerate information from MR and CT images in a single image for better representation of tumors. The fusion is implemented with one of the Dual tree complex wavelet transform (DT-CWT), Discrete wavelet transform (DWT) and principal component analysis (PCA) algorithms to investigate the best one for the application of interest. Interpolation is implemented with one of both polynomial and inverse interpolation techniques. Inverse techniques including linear minimum mean square error (LMMSE) and regularized interpolation are preferred to polynomial technique. After that, threshold segmentation is implemented to isolate the tumor region. Different evolution metrics are used such as accuracy, sensitivity , precision , specifity ,…….. are used to assess the proposed framework. Simulation results prove that the frameworking depending on DWT fusion gives the best results over the existing published techniques


I. Introduction
Image fusion is the process to gather two or more images of the same scene, to get one image with rich in details.This image has good content and easier to discover.Obtaining the images resulting from the fusion process from different instruments is an important process for use in different applications, such as medical imaging and diagnosis of the diseases.Developing fusion algorithm enhances the resulting details content of the output image, and it makes the system strong and durable to the environmental influences affecting the image [1].
Medical imaging forms are (MRI) and (CT) scan images that used to illustrate the parts of the human body.These MRI and CT images give complementary information about the organs of the human body.CT images are used for visualizing bone structures, while MRI images are used to examine soft tissue.So, the fusion of MRI and CT images is predicted to give an integrated image with more information [2,3].Segmentation is the process that divides the image into groups according to its characteristics.It is used to localize the objects and image boundaries.Several image-segmentation techniques have been used for the segmentation process.These techniques are split into three types' threshold, edge and region-based techniques.The threshold segmentation uses the histogram features and gray levels of image to define the threshold value.An edge-based image segmentation technique is performed to determine the image edges.For region-based image segmentation techniques, the seed points are determined and used in the images and a suitable region growing techniques is applied to detect the objects boundaries [4,5].Medical image segmentation is an important step for diagnosis of the diseases.The goal of medical image segmentation is to detect the region of interest (tumor area), tissue volume measurement to measure tumor growth and helping for better diagnosis the diseases.It splits the image into regions based on a deterministic description, such as segmentation of the body organs in the medical applications and brain tumor segmentation [6,7].

The Novelty
The novelty of this work is the introduction of both fusion and interpolation of multi-modality images.Different fusion and interpolation algorithms are considered and compared.This framework can be used for further diagnosis applications.

II. RELATED WORK
Several studies are used for image fusion techniques.The image fusion is used to integrate multiple images and obtain a single image with much information.El-Hoseny et al. [8].introduced a technique for medical image fusion.This technique begins with image registration and applying histogram matching to minimize the fusion artifacts.After that, the Non-sub-Sampled Shearlet Transform (NSST) is performed for fusion process and the Modified Central Force Optimization (MCFO) is performed.Finally, contrast enhancement process is performed on the fused image to enhance the contrast of the images.
Akbarpour et al. [9] introduced an attempt that is used to extract the region of interest that affects by Alzheimer disease from multi-spectral medical images.This attempt used the fusion and the segmentation methods.This attempt performs the dual-tree wavelet transform for the fusion process.The fuzzy C-means clustering is performed for classification process.
Rajalingam [10] presents a hybrid fusion technique that used combination of transform techniques for disease analysis.This technique performs two algorithms of the fusion process Dual Tree Complex Wavelet Transform-Non-Subsampled Contourlet Transform (DTCWT-NSCT).forthis technique, the input image is 256×256.NSCT is used to obtain high-pass directional sub-band coefficients and low pass subband coefficients of input images.This technique tested various multi-modality images.
In [11], the authors presented an image fusion technique.For this technique, the shearlet transform is performed to input images.The Wiener filter and Singular Value Decomposition (SVD) are performed to the low-pass sub-bands, and finally the fusion of low-and high-pass subbands is applied.
The authors of [12] introduced an attempt for the brain tumors segmentation.This attempt begins with pr-processing using Kernel sparse coding.After that the segmentation process is implemented with pixel classification method.Hence the region of interest is used for feature extraction.The type of feature is texture feature combined with kernel sparse coding.This technique was implemented on MR images and achieved high accuracy segmentation and low consumption time.
Ilhan proposed a system that is applied for brain tumor segmentation.Firstly, morphological processing was performed to enhance the MR images to aid for good image segmentation.Then, tumor region was defined d by performing the segmentation from the image.Finally, the median filter is used to remove the noised from segmented image.The threshold segmentation was used in this system [13].
Sheela et al. [14] presented an attempt for brain tumor segmentation in MR images.This attempt used region growing and morphological operation.This method begins with preprocessing process to remove the noise.And then, Dilation and erosion process are performed and Gaussian filter is used to obtain the output.Fuzzy C-Means clustering (FCM) is performed to region growing to detect the tumor region from the brain MR image.
Rao et al. [15] proposed a clustering technique for brain tumor segmentation.This technique consists of preprocessing, segmentation, feature extraction and region detection.In preprocessing operation, median filter is performed to remove the noise.The segmentation process is performed by using three methods K-means, Fuzzy C-Means and Adaptive K-means Clustering algorithms.The thresholding process is used to extract the feature with region growing and level set contouring.
Sharma et al. [16] presented a method for brain tumor segmentation that used the hybridization of differential evolution and OSTU.Hybridization is performed to enhance the efficiency of the skull-stripping by getting up the thresholding from OTSU and then the obtained value is optimized by using differential evolution.The optimum thresholding value is obtained by repeating the process for OSTU method.

III. THE PROPOSED APPROACH OF BRAIN TUMOR SEGMENTATION
Due to the utilization of different modalities in medical imaging such MR and CT modalities, we present a framework for medical image diagnosis from these modalities.It begins by image fusion for maximizing the amount of information in the obtained fusion results.Different algorithms are investigated in the fusion process.After the fusion image interpolation is investigated as a tool for resolution inhancement of images.The segmentation with a thresholding technique is implemented as the last stage.The block diagram of the proposed framework is illustrated in fig.(1).

Fig.1.The block diagram of the proposed approach
The proposed approach steps are: 1-image fusion.

Image fusion
Image fusion is the process by which multi-modality images for the same object are integrated in a single image with much more details and information content compared to both source images.The target for this process is to enhance the medical diagnosis process through the utilization of subsequent AI tools.Different algorithms have been presented in the literature for this target [8].These algorithms include Principal Component Analysis (PCA), Discrete Wavelet Transform (DWT) and Dual Tree Complex Wavelet Transform (DT-CWT).

Principal Component Analysis (PCA)
It is one of the popular fusion tools.The PCA begins by mean removal and the eigen decomposition of both images.The eigen values and eigen vector of one of the image is fed into those of the other image.After that a reconstruction process is performed [17].Figure 2 shows PCA image fusion diagram.

Fig. 2 image fusion using PCA [17]
The fused image is given using the following equation (1) : (1) Since g1 and g2 are the normalized components obtained from the eigenvector The DWT fusion depends on multi-resolution analysis of both images.After that a fusion rule is used to merge the different image subbands.Image decimation is adopted in the wavelet decomposition to eliminate any redundancy of images [18].The discrete wave1et transform (DWT) is the transform that decomposes the signal into mutually orthogonal set of wavelet scales as shown in fig. 3.

Dual Tree Complex Wavelet Transform
Dual tree complex wavelet transform (DT-CWT) works on the decomposition of images with complex wavelet basis functions.Both lowpass and highpass branches are decomposed with sort of decimation.This transfer maintains both amplitude and phase information.After the decomposition of both images, the function process is implemented [19].The advantages are simple computation, selectivity and perfect reconstruction [20].

Image Interpolation
Image interpolation can be defined as a signal synthesis process it can be implemented with different techniques.Both polynomial and inverse techniques are of such techniques.Polynomial interpolation depends on signal synthesis according to pixel neighborhood.On the other hand, inverse inperpolation depends on inverting a decimation model.Cubic spline Interpolation, linear minimum mean square error (LMMSE) and regularized image interpolation are used in this paper.

3.2.1Cubic Spline Interpolation
Cubic Spline is considered from the family of polynomial splines, it is used in many applications.Its closed form basis function is given by :  (2)

Regularized Image Interpolation
According to the regularized the solution of the equation ( 4) is can be calculated through the cost function minimization.Relation between high and low resolution images can be given through the bellow equation The generality of Q allows elimination of several restications on the interpolation process.For Q=I, the regularized solution tends to the inverse filter solution.It can be represented as follows  ̂= (   + ) −1    (9) Q is selected to decrease the 2 nd order difference energy of the estimated image.The 2-D laplacin is used to minimize the second order difference energy.The 2-D Laplacian is the most appropriate solution for the regularization operator.λ maintains the relation between fidelity of the data and smoozing of the image.The LR images are segmented into overlapping segments and the interpolation of each segment is implemented as follows  ̂, = (   +   ) −1    , (10) Where  , is a lexicographically-ordered LR block and  ̂, is the corresponding estimated HR block.Figure 5 shows the 2-D Laplacin operator.
Figure 4 shows the steps of the segmentation process.The process begins with filtering operation to remove the noise.
After that, the bounded box determines the tumor region.And then, the tumor region is only appeared in the image after segmentation and tumor outline is detected.Finally, the tumor area is detected clearly

Morphological Operation
Morphological operation are structuring process used to enhance the shape of the objects of interest while removing any type of spike noise.Let f be the image and s be the structuring element.The erosion of f by s is written as

Sensitivity
Count for the diseases pixels that the segmentation process cabable to define it [21,22].

Specificity
Estimate the part of negative which is correctly define.Also it is the define as the negative after segmentation evaluation [21,22] = TN / (TN + FP) ( F measure Precision and recall are used to calculate the F measure.High values for them means that the segmented regions are similar to the ground truth in location [22] F measure=2

DICE Similarity Coefficient
Used to measure the difference between the ground truth and the image after segmentation       Evaluation of the segmentation matrices of the resulted segmented image of PCA is not good after interpolation.so interpolation technique is not suitable with PCA fusion.
Performance of the DWT is high and the efficiency of the segmentation process is enhanced.This technique is appropriate for brain tumor images.The results are improved with using interpolation techniques and DWT fusion of Segmentation process.A comparison between the proposed approach of DWT and some stateof -the-art ones is included in table (4).It is clear that the proposed approaches has a superior performance ^() = ( −1 )[(3+) 3 −4(2+) 3 +6(1+) 3

Fig. 4
Fig. 4 The steps of the segmentation process where f is lexicographically ordered vectors of the unknown HR image, g is the measured LR image and v is additive noise.( ̂) = ‖ −  ̂‖2 + ‖ ̂‖ (5) where Q and λ are the regularization operator and parameter respectively.The derivative of the cost function is obtained as follows: ∂Ψ(f ̂) ∂f ̂= 0 = 2D t (g − Df ̂) − 2λQ t Qf ̂ (6) 2 ̂(   +   ) − 2   = 0 (7) The symbol t is a matrix transpose.to provide the minimum of the cost function, Solving for the  ̂

) 4
Simulation Results Simulation experiments are presented in this paper for the fusion and interpolation of MR and CT images.Three examples are considered in the simulation scenario.Each example comprises a CT image and MR image.Different fusion techniques are considered for comparison.These techniques include PCA, DWT and DT-CWT for the fusion process.After the two stages of fusion and interpolation, tumor segmentation is performed.The proposed framework achieves high accuracy after applying interpolation technique on the fused images.The threshold segmentation is applied after interpolation in consequence to detect the brain tumor region accurately.The proposed framework is compared with other models.It is superior to other models in determing the location of the tumor and achieving high accuracy of segmentation The efficiency of the proposed framework is estimated by using the standard metrics.Some evaluation metrics are used to evaluate the performance of the proposed framework such as sensitivity, specificity, accuracy, Jaccard, DICE, precision, F-measure, and Matthews Correlation Coffecient (MCC).The evaluation metrics are determined by the following formulas, where TP refers to True Positive case, TN refers to True Negative case, FP refers to False Positive case, and FN refers to False Negative case.Accuracy count for the image pixels ratio that are correctly classified [Precision Measures the similarity between the segmented image pixels and the original image pixels [21].=           =  +(14)

JaccardFigure 6
Figure6shows the used data set

Figures 7 ,
Figures 7,8,9 show the resulted fused images from DT-CWT, DWT and PCA and the resulted normal and abnormal images.

TABLE 1 resulted
Segmentation matrices for DT-CWTTABLE 2 resulted Segmentation matrices for DWT It is clear from the obtained results that The scenario comprising either DWT or DT-CWT fusion with cubic or LMMSE or Regularized interpolation gives the required level of segmentation accuracy.Hence the proposed approach is considered a good candidate for automated medical diagnosis.

TABLE 4
Performance of the proposed segmentation approach evaluation metrics and other approaches

TABLE 3 resulted
Segmentation matrices for PCA (14) 14 Spectral entropy resultsfor performance assessment of the proposed algorithm, the specral entropy curves are used for both MR, CT and the obtained results as shown in fig.(14)these results reveal that specral entropy of obtained results are mainly enhanced due to collection of discrimination features from both images in the fusion results.The Complexity The computation times for the whole frame work comprising fusion and interpolation are 113.14 and 62.39 sec for regularized and cubic interolation algorithms.It is clear from these times that the computation cost is acceptableCONCLUSIONThis paper introduced comparative study of using various image fusion techniques and different image interpolation techniques for brain tumor images.The image fusion techniques are performed between CT and MR images.The resulting images from interpolation and fusion are used in the segmentation process.DWT fusion technique have achieved high accuracy for the segmentation process.The results of these techniques are suitable for segmentation of medical images and performance enhancement of the segmentation process.The PCA fusion technique is not appropriate with interpolation techniques.