Skip to content

Advertisement

  • Review
  • Open Access

A Review of Point Feature Based Medical Image Registration

Chinese Journal of Mechanical Engineering201831:76

https://doi.org/10.1186/s10033-018-0275-9

  • Received: 19 September 2017
  • Accepted: 10 August 2018
  • Published:

Abstract

Point features, as the basis of lines, surfaces, and bodies, are commonly used in medical image registration. To obtain an elegant spatial transformation of extracted feature points, many point set matching algorithms (PMs) have been developed to match two point sets by optimizing multifarious distance functions. There are ample reviews related to medical image registration and PMs which summarize their basic principles and main algorithms separately. However, to data, detailed summary of PMs used in medical image registration in different clinical environments has not been published. In this paper, we provide a comprehensive review of the existing key techniques of the PMs applied to medical image registration according to the basic principles and clinical applications. As the core technique of the PMs, geometric transformation models are elaborated in this paper, demonstrating the mechanism of point set registration. We also focus on the clinical applications of the PMs and propose a practical classification method according to their applications in different clinical surgeries. The aim of this paper is to provide a summary of point-feature-based methods used in medical image registration and to guide doctors or researchers interested in this field to choose appropriate techniques in their research.

Keywords

  • Medical image registration
  • Point set matching
  • Optimization
  • Assessment
  • Application

1 Introduction

Medical image registration, i.e., aligning two medical images by finding an optimistic spatial transformation, plays a very important role in image-guided surgery and disease diagnosis. This alignment means that the same anatomical structures on two matching images correspond with each other spatially. One of the two purposes of medical image registration is to synthesize the associated information contained in multiple modality images, and the other is to comprehend images obtained from the same instrument at different times and positions. According to the nature of the registration bases, registration methods can be classified into two categories: extrinsic and intrinsic. The registration bases of extrinsic registration methods are often features attached to patients or clinical instruments. Markers, frameworks and artificial implantations are widely used bases in extrinsic medical image registration. Intrinsic medical image registration considers the information containing in the images as bases, such as the centerline of vessels and the outline of bones. Intrinsic medical image registration methods usually fall into two categories: intensity-based methods and feature-based methods [1]. Intensity-based approaches employ difference measurements based on the pixel intensities, while feature-based methods use distance-based measures of the features extracted from the input images to be aligned. Feature-based algorithms are widely employed because of their briefness, simplicity and low computational complexity over intensity-based methods. More importantly, multimodal registration using computer tomography (CT), magnetic resonance imaging (MR), positron emission tomography (PET), X-ray and other modality medical imaging methods is the main task of medical image registration. The intensity differences among the multimodal images vary considerably due to the different imaging principles, which prohibit the development of intensity-based registration methods. Feature similarities among the multimodal images are more explicit than the intensity differences, and as a result that feature-based methods play an important role in medical image registration [2].

In this review, we focus on the point features used in medical image registration, because points are the most commonly used features in clinical applications. Point-feature-based registration means that the geometrical transformation between two images is only obtained through the characteristics of the feature points. The geometrical transformation can be classified into the rigid transformation and the non-rigid transformation according to whether there are feature deformations or not. Rigid geometric transformations are often used in the registration of rigid structures, such as bones. Another important application of a rigid transformation is to obtain a rough pre-registration result before a more complex and precise geometric transformation. Non-rigid geometrical transformations are usually used to finalize the precise registration or the registration of structures accompanied with deformations.

Point acquisition is the first step towards the PMs in medical image registration. The points extracted from the medical images can greatly reduce the amount of information to simplify the computational complexity. These extracted points are usually expressed as point sets to calculate the spatial transformation. The framework in Figure 1 shows the detailed process regarding how to a convert medical image registration to a point set matching problem. The acquisition of the transformation model is a key component of point set matching. Transformation models are usually chosen according to different criteria, including the deformation of point sets, time and accuracy requirements of the registration, and the signal to noise ratio. Point set matching algorithms are the fundamental methods used to find the geometric transformation between two point sets, and have gained immense developments in image registration [3, 4].
Figure 1
Figure 1

Procedure of the PMs in medical image registration

As shown in Table 1, there have been several reviews of medical image registration methods and point set matching algorithms. Markelj et al. [1] surveyed the techniques used in 3D/2D registration methods for image-guided vascular interventions where CT, MRI, and X-ray images are the fundamental resources. In their paper in 2017, they also covered numerous available PMs as well as methods to assess them [3]. Matl et al. [2] contributed a living review of vascular image registration in which abundant vessel based registration methods were discussed from variant aspects. Oliveira et al. [5] reviewed medical image registration methodologies focusing on their geometric transformations, similarity measures, optimizers and accuracy assessment techniques. Sotiras et al. [6] conducted a survey regarding deformable medical image registration in which many nonrigid point set matching algorithms were discussed deliberately. Multi-modality medical image registration is the most commonly applied field of feature-based medical image alignment. James et al. [7] provided a joint view of different image modalities applied in medical image fusion. Point sets are more familiar in image-guided vascular interventions in which segmented vascular bifurcations, centerlines, and edges can serve as the source of feature points.
Table 1

Reviews of medical image registration and the PMs

Reference

Year

Content

[1]

2012

A review of 3D-2D registration methods for image-guided interventions

[2]

2017

Vascular image registration techniques: A living review

[3]

2017

Recent developments and trends in point set registration methods

[5]

2014

Medical image registration: a review

[6]

2013

Deformable Medical Image Registration: A Survey

[7]

2014

Medical image fusion: A survey of the state of the art

The PMs in medical image registration are efficient and time-saving, which is crucial in clinical applications. Nevertheless, a review focusing on point-feature-based medical image registration methods has not been published. The aim of this paper is to provide a summary of the PMs used in medical image registration. The remainder of the paper is organized by the following: In Section 2, the related techniques of the PMs are discussed. In Section 3, Clinical applications of the PMs are classified by the type of clinical surgery. In Section 4, a discussion and a conclusion are given to summarize the paper.

2 Related Techniques of the PMs

PMs in medical image registration depend deeply on the acquisition of feature points. There are various point acquisition methods, which are discussed in detail in Section 2.1. When feature points are obtained, the aim of the registration between two medical images is determined by the optimal geometric spatial transformation between the two feature point sets. Subsequently, three core steps of the geometric transformation are introduced in Sections 2.2 to 2.4, including transformation methods, optimizers, and an assessment of the transformation methods.

2.1 Point Acquisition

Feature points applied in medical image registration fall into two categories: extrinsic and intrinsic points. Extrinsic points are obtained by affixing certain artificial markers to patients or clinical instruments, while intrinsic points are often acquired relying on anatomical structures contained in the medical images.

2.1.1 Extrinsic Point Acquisition

Extrinsic points are usually markers attached to frames, skins, bones or organ surfaces. These points are commonly used to distinguish soft tissues when there are not enough distinguishable structures to provide reliable references [812]. The number of extrinsic points used for registration is small and in most cases, the correspondences between the extrinsic points are usually known before registration. Precise closed-form solutions optimized by a least square method are usually used to obtain the spatial transformation. However, the focus of this paper is not on extrinsic points but on the intrinsic point sets where the number of points is large and the correspondence between two point sets are usually unknown.

2.1.2 Intrinsic Point Acquisition

Intrinsic points are subsampled from features, such as centerlines, outlines, corers, surfaces, and some special points, which are obtained from the medical images without attaching markers to the patients. Consequently, the intrinsic points cause less interference to patients and surgeries. As shown in Table 2, the centerline is one of the most widely used features to obtain feature points for the rising demand of vasculature registration in minimally invasive vascular interventions (MIVI). Many vessel thinning and centerline extracting methods are used to obtain the proper centerlines. Methods such as the vessel segmentation, vasculature filter, and centerline representation [1316], have been employed to extract the centerlines in the registration of coronary or liver vessels [1721]. In other interventions, such as an abdominal aortic aneurysm (AAA) intervention [22], graph-based segmentation [23] and a sequential topological thinning process [24] are used to generate the centerline of the abdominal aorta. Centerlines also play an important role in the registration of multi-phase CT images [25], retinal images [26] and airway trees [27]. Bifurcations are special centerline points containing significant vascular information which makes bifurcation a vital component of centerline-based vasculature registration [28, 29]. Retinal image registration is a special branch of vasculature-based registration because retinal image registration is restrained to 2D-2D image registration in a relatively small and equal range. By embedding these particular properties of the retinal image into the registration process, many developed registration algorithms in computer vision and natural image registration are introduced into retinal image registration. Edge points are extracted from notable features [30] embedding the geometry information into the registration. Local features, such as corners, speed-up robust feature (SURF) points and scale invariant feature transform (SIFT) points, are also extracted to improve registration accuracy by incorporating local features into the registration [3133].
Table 2

Intrinsic point sets acquisition

Reference

Point acquisition

Source

[1324]

Centerline

CTA or XA/DSA

[25]

Centerline

Multi-phase CT

[26]

Retinal images

[27]

Airway datasets

[28, 29]

Bifurcation

3D/2D centerline

[30]

Edge point

Retinal images

[3133]

Local features

[34, 36]

Contours

X-ray views

[35]

Contours

HDR CT images

[3842]

Sub-sampling

Bone surface

For 3D-3D medical image registration, points are usually extracted from surfaces and curves of 3D models.

Contours obtained from the cross sections of 3D models are the most frequently used curves in 3D-3D medical image registration [3436]. Reconstructed surfaces, such as bone surfaces or other organ surfaces, cannot be used directly during registration because these surfaces contain abundant data, which will increase the computational complexity of the registration dramatically. Therefore subsampling methods are widely used in surface registrations to reduce the amount of data. Uniform sampling, random sampling, normal space sampling and relevance-based sampling [37] are the most commonly used subsampling methods to acquire point sets efficiently. The automatic surface sub-sampling method is extensively used in the US and CT bone surface registration [3841]. Manually picked sampling is an effective but time-consuming approach, which is also used in the registration of US, MR and CT bone surfaces [42].

2.2 Geometric Transformation Methods

The geometric method registers two images by minimizing a distance calculation criterion. Before describing any methods, we introduce the known and unknown variables of the problem. The rigid transformation between the target point set (M) and the source point set (S) can be expressed as follows:
$$M = R*S + t$$
(1)
where R and t represent the rotation and translation parameters, respectively, in the transformation. If M and S are 3D point sets, then R and t represent the three rotation parameters and three translation parameters respectively. While in the 2D-2D rigid registration, there are only four parameters comprising two rotation parameters and two translation parameters. The iterative closest point (ICP) algorithm (Section 2.2.1) is the most classical and prominent method for rigid point set registration. However, it is difficult to describe the complex correspondence between M and S when there are deformations. The following formula only describes the criteria for non-rigid registration:
$$M = f(S,\theta )$$
(2)
In this formula, f represents the transformation model, and θ represents parameters in the transformation model. The robust point matching (RPM) algorithm (Section 2.2.2) is proposed using a spline function as the transformation model. The graph matching (GM) algorithm (Section 2.2.3) is proposed when f is described as a method to match two graphs. The minimized energy function (MEF) method (Section 2.2.4) is also widely used in describing deformations incorporating the energy function into the transformation model. The Gaussian mixture model (GMM) (Section 2.2.5) and the coherent point drift (CPD) algorithm (Section 2.2.6) are proposed using the distribution of two point sets to model the difference between the two point sets. The distance transformation (DT) method (Section 2.2.7) shows several special distance measurements, which are more effective than the Euler distance measurement, in describing the difference between two point sets. The application of these transformation models in clinical surgery is shown in Table 3.
Table 3

Application of PMs in different surgery

PMs

Surgery

Vasculature intervention

Retinopathy

Laparoscopic surgery

Orthopaedics

Others

ICP

[20, 29, 47]

[26, 28, 33]

[81, 82, 84]

[34, 36, 38, 39]

[78]

RPM

[49]

[31]

[35]

GM

[29]

[28]

GMM

[68]

[30, 41, 42]

[40]

CPD

[25]

[71]

DT

[18, 19]

[40]

[83]

MEF

[17, 21, 22, 61]

2.2.1 ICP

The ICP algorithm, introduced by Besl and McKay [4], considers that the target point set (M) can be registered to the source point set (S) interactively after a rigid transformation. The fundamental principle of the ICP algorithm can be expressed as follows:
$$dist(R,t) = \frac{1}{N}\sum\limits_{i = 1}^{n} {||Rm_{i} + t - s_{i} ||^{2} }$$
(3)
where \(m_{i} \in M\) and \(s_{i} \in S\). M is aligned to S, while \(dist(R,t)\) reaches the minimum value. The ICP algorithm has been significantly developed. Many variants of the ICP method have been proposed to affect all phases of the algorithm. For error minimization, the classical ICP algorithm uses the Euclidean distance to construct a quadratic objective function subjected to a nonlinear constraint, while Sebastien developed a variant of the ICP algorithm called EM-ICP by applying the EM principle and an annealing scheme to avoid local minima [43]. Fitzgibbon proposed the LM-ICP method [44] where the objective function of the distance error was optimized with the Levenberg–Marquardt algorithm [45]. The ICP algorithm is simple and has a low computational complexity, which also supports parallel computing. Nevertheless, the ICP algorithm suffers from its accurate demands of an initial estimation and limited application for only rigid transformations considerably. Another drawback that confines the application of the ICP approach is that the local minimum is obtained ultimately instead of a global minimum, even though Yang et al. [46] introduced a globally optimal solution Go-ICP based on the branch-and-bound algorithm under the L2-norm closest-point error metric. Another application is to combine the ICP algorithm with some global algorithms to obtain a global minimum [19, 29]. However, it is difficult to balance the speed and accuracy of the calculation.

Due to the characteristics of the ICP algorithm, it has been widely used in the rigid registration of medical images, such as the registration of 3D reconstructed rigid surfaces[34, 36, 38, 40] and vasculature without regarding the heartbeat and breathing[26, 28, 33, 47]. Nevertheless, the ICP method is not suitable for some non-rigid registration especially with large deformations.

2.2.2 RPM

The robust point matching method (RPM) was initially put forward to simultaneously find the correspondence and transformation parameters between two point sets [48]. A deterministic annealing method was used to guarantee the convexity of the error function of the RPM. The RPM uses a coordinate descent approach to calculate the transformation parameters and a soft-assign algorithm to estimate the correspondence between two point sets. Many variants of the RPM are proposed to modify the phases of the method from establishing a correspondence to optimizing the transformation parameters. Chui et al. developed the TPS-RPM [49] algorithm by using a thin plate spline (TPS) to parameterize the non-rigid mapping procedure because the TPS can describe affine and non-affine deformation elegantly. Zheng et al. [50] developed the RPM-LNS method by employing a simple graph to note the neighborhood structure of the point sets, which can transform the alignment of two point sets to the optimal matching between two graphs. The RPM-LNS method is generalized by the TPRL algorithm which relaxes the labeling process with an optimal compatibility coefficient. The Shape context is also an extensively employed descriptor to present the correspondence between points among the point sets [5154].

As a non-rigid point set matching algorithm, the RPM has obtained several successes in the registration of medical images with small deformations, such as retinal images [31] and high-dose-rate (HDR) CT images for gynecological cancer detection [35]. As is well known, the quality of medical images is unsatisfactory due to the poor imaging mechanism, which substantially suffers from the noises of surgical instruments and the physiological activities of patients. The RPM performs poorly during the registration with high noise and outliers, which limited the application of RPM dramatically.

2.2.3 GM

Graph matching (GM) based approaches treat point sets as graphs. GM generally builds an adjacency matrix to present the correspondence between two graphs, where the nodes represent the correspondence between points and the weights stand for the pairwise agreements between potential correspondences [55]. The correct assignments of the matching points are judged by whether the matching pairs are able to form a strongly connected cluster of the adjacency matrix. The graph shift (GS) method was proposed to eliminate the drawbacks of the GM approach to reduce its sensitivity to noises and outliers [56]. Subsequently, numerous algorithms have emerged to improve the robustness or computational complexity of the GM approach [57, 58]. Sousa et al. [59] combined the advantages of GM with the CPD method by adding the topological contribution of the GM into the CPD method to develop a stronger method.

GM is a proper approach to describe the topological structure of some vessels in which bifurcations can directly represent the nodes of a graph. Consequently, in medical image registration, GM has been mainly employed to register medical images with clear topological structures [28, 29].

2.2.4 MEF

The energy method was first proposed for mechanics [60] and now has been increasingly used in more fields. For deformable medical image registration, a similarity measurement is frequently presented as an energy function, which incorporates the deformation information and no deformation information of two images. The energy equation of the MEF usually consists of two items: the internal energy item and the external energy item. For vessel-based coronary image registration with the constraint of temporal continuity [20], the external energy Eexternal is generated by the transformation between the two images to be aligned, while the internal energy Einternal represents the information of only one point set, including the displacement EDisp, the smoothness ESmooth and the myocardium constraint EMyocard. As mentioned above, the total energy function can be expressed as
$$\begin{aligned} E(r) &= E_{external} (r) + E_{Disp} (r) + E_{smooth} (r) + E_{Myocard} (r) \\ &= \sum\limits_{i = 1}^{K} {H(x_{i} )[\hat{x}_{i} - (x_{i} + r_{i} )]} + \mu \sum\limits_{i = 1}^{K} {|r_{i} |^{2} } + \beta \sum\limits_{i = 1}^{K} {|\dot{r}_{i} |^{2} } + \gamma \sum\limits_{{\{ i,j \in K\} }} {\left (\frac{{|r_{i} - r_{j} |}}{{|x_{i} - x_{j} |}}\right)^{2} } \\ \end{aligned}$$
(4)
where r is the transformation parameter, xi is one of the centerline points in the segmented fluoroscopy images, \(\hat{x}_{i}\) represents the centerline point of the model, \(H(x) \in \{ 0,1\}\) is used to reject the outliers and μ, β and γ are energy-balancing free parameters.

To describe the deformable registration of vascular structures, an energy function was proposed to map the deformable registration process [61]. The external item of this energy function is a difference measurement between two point sets, and the internal energy consists of a length preservation item and a smoothness preservation item. This method is also used in an efficient graph-based 2D/3D deformable image registration algorithm, which is applied in abdominal aortic aneurysm interventions [62].

2.2.5 GMM

The Gaussian mixture model (GMM) algorithm for point set registration was proposed by Jian et al. [63]. These authors used the Gaussian mixture model to describe the distribution of the two given point sets M and S and treat the registration of two point sets as the alignment between two Gaussian mixtures by minimizing their L2-distance. The principle of GMM can be described as follows:
$$d_{{L_{2} }} (S,M,\theta ) = \int {(gmm(S) - gmm(T(M,\theta )))^{2} {\text{d}}x}$$
(5)
where θ stands for the transformation parameter of S to align with M. The expectation maximization (EM) [64] method was employed to minimize the L2-distance and maximize the kernel correlation of the point sets. Although widely applied, the EM algorithm is quite sensitive to initial values, and its component number need to be pre-determined. To eliminate these drawbacks of EM, Yang et al. proposed a robust EM clustering algorithm which uses all the data points as the initialization. The authors have overcome the difficulties of choosing a initial values and constructing a competition schema by embedding a penalty term into the EM algorithm [65]. To improve the accuracy of the GMM for 3D rigid/non-rigid registration, a convex hull indexed Gaussian mixture model (CH-GMM) [66] was proposed by incorporating the proximity area conservation and projection consistency into a weighted GMM model. Dylan et al. [67] did not represent the distribution of the point set as a GMM directly but approximated the output function by training a one-class support vector machine (SVM) by a GMM. This method exploits the robust correspondence presentation of the SVM with sparse parametrization and occlusive outlier rejection to improve the robustness of the algorithm and minimize the L2-distance between two support vector-parameterized Gaussian mixtures.

As a probabilistic method, the GMM has achieved excellent results in medical image registration where the number of feature points is considerable. The feature points in medical images of vessels are sufficient, because the centerlines and edges can be directly used as the source of the feature points. Consequently, the GMM is widely used in the registration of medical images of vasculature [68] or retinopathy [30, 40, 41]. However, the GMM relies greatly on the initial position of the GMM centers, which restrict the application of GMM and affect the registration result severely. As a global registration algorithm, another deficiency of the GMM is that it does not incorporate the detailed information among the feature points into registration, thus the registration accuracy decreases greatly when the detailed information plays an important role in the alignment procedure. The most challenging task of the GMM is that a specific component number should be selected in advance, in which expert experience and a pre-analysis of the images to be aligned are needed.

2.2.6 CPD

The coherent point drift (CPD) algorithm, a probabilistic method extended from the GMM for both rigid and non-rigid registration of two point sets, considers the registration of point sets as a maximum likelihood estimation problem which can be described as follows:
$$\begin{aligned} Q(\theta ,\sigma^{2} ) &= \frac{1}{{2\sigma^{2} }}\sum\limits_{n = 1}^{N} {\sum\limits_{m = 1}^{M} {P^{old} (m|x_{n} )||x_{n} - {\rm T}(y_{m} ,\theta )||^{2} } } + \frac{{N_{P} D}}{2}\log \sigma^{2} \\ \end{aligned}$$
(6)
where σ2 is the equal isotropic covariance of the GMMs and θ represents the transformation parameters, X = (x1, …, xn)T is the data point set and Y = (y1, …, yn)T represents the GMM centroids [69, 70]. The author regards the motion of the Gaussian centroids from the initial position to their final position as a temporal motion process. To preserve the topological structure of the point sets, the GMM centroids are restricted to move coherently as a group. Although the CPD approach performs more accurately and robustly than its competitors in the registration of two point sets with noise, outliers and missing points, it also has two shortcomings. One issue is that the weight parameter w that estimates the level of noise and number of outliers in the GMMs is manually selected, and the other issue is that the CPD method does not take the neighborhood information of the points in the same point sets into account. To overcome these two shortcomings of the original CPD algorithm, Peng et al. [71] proposed a robust CPD algorithm, which uses the shape context [5154] to describe the neighborhood structure of the points and employs the EM framework to calculate and optimize w automatically. The robust CPD algorithm is a great improvement of the traditional CPD approach, where detailed information is incorporated into the registration procedure. To accelerate or enhance the CPD algorithm, numerous variants of the CPD approach are proposed focusing on the different phases of the algorithm [7275].

Similar to the GMM, CPD is also a probabilistic method in which it is difficult to describe the local neighborhood structure of the feature points. Although many local similarity descriptors have been proposed to eliminate the drawbacks of the CPD methods, there are still great difficulties in describing the complex anatomical structures as well as the local deformation of the medical images. Despite the fact that applications in medical image registration are restricted by the drawbacks mentioned above, the CPD method is still successfully applied in the registration of multi-phase coronary CT images [25] and adjacent CT slices [71].

2.2.7 DT

The distance transformation (DT) method does not use the traditional L2-distance but some novel and ingenious distance measurements to calculate the difference between the two point sets. Here, we summarize two typical DT algorithms used in medical image registration.

To calculate the distance between the projected coronary CTA centerlines C and the X-ray images X, a sigmoid-like function was proposed by expanding the Euclidean distance [18]. The distance measure can be described as:
$$S(u,v) = \sum\limits_{k = 1}^{n - 1} {\frac{1}{|x|}\sum\limits_{x \in X} {h(D(X,C(u,v)))} } ,$$
(7)
$$h(d) = 2 - \frac{2}{{1 + \exp ( - {{d^{2} } \mathord{\left/ {\vphantom {{d^{2} } {h_{a} }}} \right. \kern-0pt} {h_{a} }})}},$$
(8)
where u = (s, r) represents the offset and scaling parameter, v represents the transformation parameters which vary according to the transformation types, ha is a control parameter, and D represents the function to calculate the Euclidean distance. For the same purpose, Baka et al. proposed the G1/G2 distance measure for 2D-3D vasculature registration [17], in which the distance measure is a composition, including a 2D Euclidean measure, an orientation difference and a Frangi vasculature score Fj. the distance measure can be expressed as follows:
$$\theta_{ij} = \alpha (1 - F_{j} ) + \left( {1 - |\cos \gamma_{ij} |\exp \left( { - \frac{{D_{ij}^{2} }}{{\sigma^{2} }}} \right)} \right),$$
(9)
where Dij is the 2D Euclidean distance between the projected vessel point i and the centerline point j, γij is the angle between the 2D centerline direction and the projected 3D vessel direction, and α is a parameter determining the influence of Fj. There are also several other DT methods modified from the Euclidean distance [19, 21] or other point set matching approaches [29].

2.3 Optimization of the Geometric Transformation

The iterative point set registration problem is a fundamental optimization problem to search for the maximum or minimum value of the similarity measurements. As shown in Table 4, according to the objective function as elaborated in Section 2.2, we classify the optimization methods into two categories: an unconstrained nonlinear optimizer and a statistical optimization model.
Table 4

Optimizers used in PSM-based medical image registration

Classification

Reference

Optimization

Unconstrained nonlinear optimizer

[8, 21]

Gradient descent

[18, 40]

Non-linear least squares optimizer (NLS)

[19, 20]

Powell-Brent optimizer

[22, 27]

[61]

Broyden–Fletcher–Goldfarb–Shanno (BFGS) optimizer

[29, 36]

Downhill simplex algorithm

[33]

Levenberg-Marquardt (LM)

[38]

Gauss-Newton framework

[17]

Nelder–Mead optimizer

Statistical optimization model

[25, 35]

[71]

Expectation-maximization (EM)

[34]

Minimum description length (MDL)

[42]

Barnes-Hut algorithm

[67]

Support vector machine (SVM)

The correspondence of the point-to-point registration is established based on each single point pair. Several unconstrained nonlinear optimizers have been used to determine the correspondence between two point sets, including the non-linear least squares optimizer (NLS) [17, 39], Levenberg–Marquardt (LM) algorithm [33], downhill simplex algorithm [29, 34], Nelder–Mead optimizer [21], Gauss-Newton framework [38], Gradient descent [8, 20], Powell-Brent optimizer [18, 19], and Broyden–Fletcher–Goldfarb–Shanno (BFGS) optimizer [22, 27, 61]. The references that contain detailed descriptions of these algorithms are displayed in Table 4. The applicable conditions and selection criteria of these optimize algorithms are elaborated in detail in the book by Press et al. [76] if the distance measure between two point sets has been modeled as a convex function. If the distance measure is non-convex, then Khoo et al. provide a proper method to convert a non-convex function into a convex function to improve the registration efficiency by relaxing some constrains [77].

Several probabilistic registration methods, whose correspondence is set up on the distribution of points, cannot be optimized by traditional optimization methods. Some statistical optimization models have been applied to optimize these point set matching algorithms such as the GMM and the CPD method. The minimum description length (MDL) [36], Barnes-Hut algorithm [41], expectation-maximization (EM) [25, 35, 71] and support vector machine (SVM) [67] are presented as methods based on the point set distribution.

In summary, an optimization method satisfies a certain objective function. We only provide a general direction for the selection of optimization methods. Specific optimization method should be chosen after referring to the professional books [76].

2.4 Assessment of the Geometric Transformation

A registration has little value unless its accuracy can be evaluated. Several approaches have been proposed to assess the registration accuracy, which is shown in Table 5. Various distance measures have been used in the PMs of medical image registration, so the evaluation method is usually a certain error function to assess the registration dissimilarity between the two point sets.
Table 5

Registration accuracy assessment methods

Reference

Evaluation method

[8, 18, 21, 27, 29, 31] [17, 35, 36, 39, 47, 61, 71, 78, 85, 87]

Root mean square error(RMS)

or

Mean square error (MSE)

[20]

Residual error and standard deviation

[22, 3841]

Target registration error (TRE)

[28, 33]

Success rate(SR)

[30, 31]

Median error (MEE), Maximum error(MAE) and RMS

[41, 42]

TRE and Surface registration error (SRE)

The root mean square error (RMS) or mean square error (MSE) has been the most commonly used approach to evaluate the registration results. These criterions effectively evaluate the registration between two point sets subsampled from the vessel centerlines or bifurcations in vasculature interventions [17, 20, 21, 29, 31, 47, 61], contour or surfaces of bone in orthopaedics [34, 42], edge points and some other local feature points of multiple kinds of medical images[8, 27, 35, 71, 78], because these criterions can take advantage of the distance measures directly without introducing additional errors. However, the drawback of the RMS is also its dependency on the distance measure. When the distance measure is not chosen properly, the RMS will rise dramatically, which will reduce the value of the evaluation criteria. The target registration error (TRE) calculates the distance between the corresponding points that might represent surgically targeted tissues. These approach is an objective and accurate measurement, which is applied when there are known ground-truth positions of the validation points [22, 38, 39, 41, 42]. The success rate (SR) is a unique evaluation in the PMs by counting the successfully registered point pairs. The SR has been applied in retinal image registrations to assess the successful registration rate of centerline points or bifurcations of retinal vessels [28, 33].

Some methods use more than one evaluation method to obtain a more convincing result. The FRE and SRE are used to estimate registration accuracy between point sets subsampled from CT and US bone surfaces [40, 41]. The residual error and the standard deviation are chosen to evaluate registration between a 3D coronary projection and XA images, where the pixels of the vessel are considered as the point sets to be aligned [19]. The MEE, MAE, and RMS work as standards when the authors compared their proposed method with many earlier algorithms to confirm that their method exhibited a more persuasive performance in the registration of retinal images [30, 31].

Robustness, speed, convergence, and stability are also frequently used to evaluate and compare the performance of numerous PMs [3]. The design of the evaluation image data sets, the definition of the corresponding ground truth and accuracy, the selection of evaluation criteria, the design of evaluation metrics and the design of the evaluation protocol are prerequisites in the standardization of medical image evaluation methodology [79]. However, a gold evaluation criterion to assess the performance of the PMs in medical image registration methods has not been published.

3 Clinical Applications of the PMs

Medical images vary considerably in different clinical surgeries. The previous section provides a comprehensive view of the PMs used in medical image registration. However, PMs are also influenced by the various circumstances in different clinical surgery. For vascular intervention (Section 3.1), the urgent requirement is the registration between 3D CTA/MRA models to the 2D X-ray/DSA images. For retinal image registration (Section 3.2), time and position are the prominent differences between two retinal images. Although the registration of bones is rigid, the modalities of the medical images are complex in orthopeadic surgery (Section 3.3), including ultrasound (US), CT and X-ray images. The PMs are also applied in some other clinical applications (Section 3.4) such as the registration of airways, ventricle surfaces, lung surfaces or a 3D reconstruction of CT or MR slices.

3.1 PMs in Vascular Intervention Surgeries

Enhancing interventional visualization is an essential part of image-guided interventions by integrating different modality information extracted from intraoperative and preoperative images into one single coordinate framework [1]. Combining complementary information taken from various imaging modalities or taken at the different times helps the surgeons obtain a comprehensive awareness of the vasculature and provides a better diagnosis of diseases. The 3D computed tomography (CT) or magnetic resonance images (MRI) usually acquired as pre-interventional data, and 2D X-ray projected images are typically collected as the intro-interventional data. The alignment between CT/MRI and X-ray images forms a 3D/2D image registration scheme.

There are abundant classical methods to classify 3D/2D vasculature registration, as described in a living review developed by Stefan Matl et al. [2]. The most commonly used methods are pixel or voxel similarities based similarity measurements and feature based transformation estimations. The intensity information between the different modality images diverges greatly from each other, while the anatomical structures of vessels, such as centerlines and bifurcation points can demonstrate the dissimilarity between the two images more competently. Meanwhile, features after segmentation contain less data, which can reduce the computational complexity dramatically. Features used for vasculature registration are often geometrical entities such as isolated points or point sets sampled from curves, contours or surfaces. Consequently, the basis of most feature-based registration methods is point set registration. Point set registration of vasculature generally consists of the following five parts:
  1. 1)

    Segmentation and preprocessing of 3D and 2D images,

     
  2. 2)

    Centerlines or edges extraction,

     
  3. 3)

    Point resampling of the centerlines or the edges,

     
  4. 4)

    Point preprocessing,

     
  5. 5)

    Minimizing the distance between two point sets.

     
The first three steps of the five parts mentioned above are shown in Figure 2, which shows two approaches to convert the 3D-2D abdominal aorta image registration to align two 2D vascular centerlines that can be used as point sets directly in the PMs. The PMs applied in vasculature interventions vary seriously in different clinical applications. Table 6 at the end of this paper shows an overview of the relationship between The PMs and their clinical applications.
Figure 2
Figure 2

Point set registration of an abdominal aorta. (a) DSA, (b) reconstructed 3D CTA model, (c) binarization of the DSA, (d) DRR image, (e) 3D centerline, (f) centerline of DSA, (g) centerline of DRR, (h) projection of 3D centerline

Table 6

Summary of PMs and their applications in medical image registration

Reference no.

Point acquisition

Source

Distance measure

PMs

Optimize strategy

Evaluation method

Surgery

[18]

Centerline

CTA/ DSA

G1/G2

NLS

RMS

Coronary intervention

[19]

Centerline

Skx(µ; ν)

Powell-Brent

Projection error

[20]

Pixels of vesselness

DT-basedICP

Powell-Brent

Residual error and standard deviation

[21]

Centerline

Euclidean distance

Minimize energy function

Gradient descent

RMS

[25]

Centerline

Group-wise CPD

EM

RMS

[68]

Centerline

 

L2 distance

OGMM

Median accuracies

[22]

Centerline

CTA/ DSA

GM-MEF

BFGS

TRE

AAA

[61]

Bifurcation

CTA/ DSA

Penalized Euclidean distance

MEF

BFGS

RMS

[26]

Centerline

Retinal images

L2 distance

DB-ICP

EM

RMS

Retinopathy

[28]

Bifurcations

L2 distance

GM-ICP

SR

[30]

Edge point

L2 distance

GMM

EM

MEE, MAE, RMS

[31]

SURF points

SURF–PIIFD–RPM

[32]

Harris corners

Harris-PIIFD

MAE

[33]

Corner points and face points

GDB-ICP

LM

SR

[27]

Centerline

Airway datasets

Tensor Fields

BFGS

MSE

Respiratory

[29]

Bifurcation

CTA/ DSA

Ficp

ICP + Graph based

Downhill simplex

RMS

Liver interventions

[17]

Centerline

 

EDT

Nelder–Mead

Distance errors

[34]

Contours

X-ray views

L2 distance

ICP

Down Hill Simplex

Orthopaedics

[36]

Contours

L2 distance

ICP

MDL

RMS

[38]

Randomly generated

Bone surface meshes

IMLP

Gauss-Newton

TRE

[39]

Sub-sampling

US/ MR/CT

Stochast-ICP

Simulated annealing

RMS and TRE

[40]

Sub-sampling

US / CT

Euclidean distance

NLS

TRE

[41]

LGF

L2 distance

GMM based

Barnes-Hut algorithm

SRE and TRE

[42]

Sub-sampling

L2 distance

GMM

GPU

FRE and SRE

[35]

Sub-sampling

Segmented CT

TPS-RPM

EM

RMSE

Gynecological cancer

[47]

Centerline

CTA/ DSA

 

Variant of ICP

 

RMS

Neuro-intervention

[71]

Contours

CT /MRI slices

Robust CPD

EM

RMSE

Reconstruction

[81]

Surface points

Kidney surface

 

ICP

 

TRE

Laparoscopic surgery

[82]

Liver surface

 

ICP

 

RMS

[83]

Liver surface

DT

  

FRE

[84]

Tissue surface

 

ICP

 

TRE

[90]

SIFT feature points

MRI slices

KFM-PDM

MSE

Reconstruction

In coronary intervention, the biggest challenge is that coronary vessels move along with a beating heart. Therefore the registration of coronary images must incorporate the movement of the heart into the procedure. Nonrigid registration for large deformations exactly fits this condition. Baka et al. [68] extended the original GMM algorithm for 3D-3D point set registration to the 3D-2DOGMM algorithm incorporating vascular orientation into the alignment of the preoperative computed tomography angiography (CTA) and the intraoperative X-ray angiography of coronary interventions. The point sets in OGMM were obtained from the coronary centerline points, which appropriately represent the topological structure of vessels. The distance measurement calculating the difference between two point sets is adjusted in this model to achieve a more accurate result. The OGMM algorithm first incorporates the distinctive features of vessels into the registration process. This improvement causes the OGMM to outperform the ICP algorithms, G1 and other distance transformation algorithms [1720] in accuracy and convergence rate. Another procedure to improve the performance of the OGMM is to modify the optimization process. Khoo et al. [77] developed a convex programming framework to optimize the correspondence and the relative transformation between the point sets simultaneously. This effective framework can obtain a global optimum by transforming a non-convex process to a convex process by relaxing some restrictions of the optimization. By incorporating some priori information, such as transformation invariant features and gradient information, into the preprocessing of registration, the author concluded that the relaxed convex version converges to the solution of the original program despite the sharp sensitivity to noise. Combining this algorithm with some local information based methods takes advantages of both local and global methods to obtain a more accurate optimum. Motivated by the application of coronary matching in multi-phase cardiac spiral CT, a group-wise CPD algorithm [25] was proposed that embed the temporal consistency constraints between the point sets into registration to overcome one of the biggest problems of multi-phase point set registration. The point sets are obtained from manually annotating centerlines and branch points. The coronary vessels deform sharply as the heart beats, so the registration between the multi-phase CT is a nonrigid process where the CPD algorithm performs effectively. Although the algorithm has achieved elegant results in the registration of multi-phase coronary vessels, it is very restrictive with too many artificial interventions.

Liver vasculature intervention is another vital surgery to detect liver cancer, liver biopsy, liver cirrhosis, portal hypertension and other liver diseases. Kim et al. presented an EDT [80] based algorithm to register the chosen relevant 3D vessels of CTA and the given DSA image [21]. The EDT method was applied in this algorithm to calculate the distance between two point sets extracted from the projected CTA images and DSA images. The 3D vascular structure was divided into several segmented subtrees which ensure that the given DSA is aligned with the most similar subtree. Groher et al combined ICP with GM to obtain a novel 2D-3D liver vasculature registration algorithm. This combination takes advantage of the low computational complexity of ICP and the topological information of GM, which helps the registration process converge to an optimum result quickly and robustly [29].

Abdominal aortic aneurysm intervention (AAA) is also influenced severely by heart beating. The diameter of aorta is larger than the liver and the coronary vessels. Additionally, there are fewer branches in the aortic vessels. Consequently, the most commonly applied PMs in AAA intervention is MEF [22, 61], which not only describes the large deformation but also preserves the smoothness and length features of the aorta.

Deformation of cerebrovascular tissue in neuro- intervention is so slight that can be ignored, so medical image registration in neuro-intervention is generally considered to be a rigid registration. The variants of ICP are efficient to align two cerebrovascular images [47].

PMs utilized in vascular intervention surgery as mentioned above, have gained significant achievements in some specific applications. Nevertheless, there are some deficiencies in these methods, which limit their clinical applications. First, the viewable range of CTA and DSA images differs greatly, which increases the rate of missing points and noise of the registration and enlarges the dissimilarity between two point sets. Consequently, the fit of PMs for point sets with large number of differences should be developed to eliminate this restriction. Second, only the centerline points or bifurcation points are used in the registration process, other vessel information including diameters and curvatures are rarely considered in the proposed papers. Future algorithms should take these features into consideration. Finally, all the algorithms mentioned above could not describe the deformation of vessels accurately, so future algorithms should be more capable of describing the deformation of the vascular images.

3.2 PMs in Retinal Image Registration

Retinal images play an important role in the diagnosis of multiple eye diseases such as diabetic retinopathy, age-related macular, degeneration and glaucoma. Multiple imaging protocols have been applied to obtain images of the different parts of the fundus. Fundus angiography is a better technique to observe blood flow, obstruction and other vascular lesions [26]. It is necessary to register multiple retinal image fragments or retinal images in multiple modes to obtain an entire retina image of a patient, to comprehend retinal image information in multiple modes and compared the vascular changes of a specific patient at different times.

Branches and intersections are the key information in accordance with the special structures of retinal images. However, there are so many branches and intersections that they are very similar to each other. The retinal lesions or poor quality images block part of the vasculature. The brightness of retinal angiography is altered greatly at the different imaging stages. All the constraints above increase the difficulty of retinal image registration. Contraposing these problems, the above researchers have proposed a variety of methods.

The dual-bootstrap ICP [26] was proposed by Stewart et al. which takes advantage of not only branches and intersections, but also some local information with high precision and similarity. Point sets are automatically abstracted from the given retinal images. The dual-bootstrap step calculates the covariance matrix of the estimated transformation to initialize and modify the transformation parameters, while the ICP step utilizes the robust ICP algorithm to minimize the distance between two point sets. The combination of the dual-bootstrap and ICP eliminates the drawback of the ICP that calls for an accurate initialization. The GM method is an excellent approach to make full use of the vascular topological structure information for registration. Deng et al. introduced the GM-ICP algorithm [28] which combined the GM and ICP to efficiently find the global correspondences between the vascular bifurcations. However, the GM-ICP is deeply affected by the sufficiency of consistent structures, which limits its application in medical image registration. Another technique [30] incorporating both local and global information of the retinal image is the Lo-GMM which integrates local image details into a GMM model (Lo-GMM). This method can effectively obtain a global result by aligning the probability density distribution of the vascular edge points in two retinal images. The local image features contain abundant information which can improve the registration accuracy greatly, but the computational complexity of the algorithm increases sharply with embedding of too much local information.

Researchers attempted to use invariant features, such as the partial intensity invariant feature descriptor (PIIFD), and the speed up robust features (SURF), of retinal images to avoid the detection of vascular bifurcations. The PIIFD and Harris corners are used to obtain point sets in the PIIFD-Harris algorithm [32]. Corner points, instead of bifurcations, serve as control point candidates which reduce the computational complexity and improve the automation of the registration procedure. Wang et al. proposed the SURF–PIIFD–RPM method combining the invariant features with RPM algorithm to register retinal images with deformations automatically [31]. Poor quality and nonvascular images, which are not considered in the PIIFD-Harris method, are also incorporated in the experiments of the SURF–PIIFD–RPM. This method takes advantage of the RPM in outlier rejection to improve the robustness of the algorithm, so that desired result can be obtained, even when the overlapping area between two retinal images is very small.

Retinal image alignment is restricted to 2D-2D registration due to imaging principles and medical applications. Different from vessel registration in vascular interventions, features applied in retinal image registration are not only branches, intersections, centerline points and edge points, but also invariant features, such as Harris and SURF features. The view range of the retinal images is exactly the same as each other, which facilitates the registration by ignoring scaling parameters. Future development should consider improving the registration accuracy as well as reduce the computational complexity when the overlapping areas are insufficient.

3.3 PMs in Orthopaedic Surgery

Registrations of bones in orthopaedic surgery are often rigid registrations. Feature points used for registration are usually derived from bone surfaces which are visible in both the reconstructed bone model and the target anatomy during surgery. Ultrasound (US) images, which do no harm to patients and surgeons, are gradually used in orthopaedic surgeries to assist the registration of preoperative and intraoperative images. US is safe and inexpensive and can be used in real-time. However, ultrasound also suffers from its sensitivity to noise and limited fields of view. High resolution and high signal to noise ratio of US make it the most commonly used methods in clinical operations despite its high price. Registering tracked intraoperative US images with preoperative CT data has been proposed as an effective mechanism for computer assisted orthopaedic surgery (CAOS) [41]. Feature points extracted from bone surfaces are frequently used to generate data sets in the pre-procedure of the PMs. Consequently, the point matching algorithms play an important role in the registration of US and CT images. Other modalities of medical images are also applied to increase the amount of useful surgical information and decrease the cost of surgery.

StochastICP [39, 42], a robust variant of ICP, has been proposed to register point sets derived from femur and skull surfaces. The robustness and precision of the stochastICP are improved greatly compared to the traditional ICP algorithm, however, stochastICP calls for a manual adjustment to obtain a proper initial position. The iterative most-likely point (IMLP) method, another variant of ICP [38], is used for the registration of human hip and femur meshes. Real-time US and CT registration is a key component of orthopaedic surgeries. Brounstein et al. [40] introduced the GMM algorithm to the registration of two point sets extracted from local bone phase images. The largest innovation of this method is that these points are chosen on the basis of local curvatures, which ensure this method to obtain a more accurate global optimum. The GMM method also demonstrates a high calculation speed to meet real-time requirements and is a good descriptor of bone surfaces for incorporating the local curvature of points. There are also many applications of the PMs in other bone-related surgery processes, such as 3D reconstruction of bones by fluoroscopic images and alignment of X-ray bone slices [34, 36].

3.4 PMs in Laparoscopic Surgery

Laparoscopic surgery (LS) is another interventional technique to access the abdominal anatomy of the patient. LS is minimally invasive in contrast to the traditional open surgery which involves cutting the skin and dividing the underlying tissues to gain a direct access to the surgical target. However, LS can only provide the surgeons with a restricted and small view of the surgical field, which abandoned the navigation of the surgery. To improve the visualization capability of LS, medical image registration is required to comprehend the various images taken in pre- or intra-laparoscopic surgery. Makers play an important role in the majority of automatic approaches for registering the endoscopic images, where rigid PMs, such as ICP and its variants, are widely used to obtain an accurate registration. Benincasa et al. [81] used ICP to register the surface points of the kidney to enlarge the view of the intraoperative surface. In image guided liver surgery, Cash et al. [82] used the ICP algorithm to register the preoperative and intraoperative representation of the liver surface in the rigid registration procedure. Matching surface correspondences is one of the key components of shape-based guidance, and Santos proposed an approach based on the distance error to establish the surface correspondences of the liver surface reconstructed from volume scanners, such as CT and MRI, and 3D range scanners to initialize the fine surface matching algorithms for intra-operative soft tissue registration [83]. PMs also ensure that optical techniques for 3D surface reconstruction procedure are effective, even when there are thousands of feature points [8487].

3.5 PMS in Other Surgical Applications

Registration of airway trees is a unique procedure where the orientation and thickness of the airway trees cannot be ignored. Wassermann et al. [27] introduced the tensor field into the registration of point sets collected from airway trees where each point is associated with a symmetric matrix representing the orientation and thickness of the airway trees. The tensor field, which is represented sparsely, is transformed from a shape feature matrix to convert the alignment of two images to the registration of two tensor fields. The tensor field algorithm is tested in both synthetic and human airways to validate that its registration accuracy is superior to GMM and other up-to-date algorithms.

Almhdie [78] proposed a comprehensive ICP (CICP) algorithm to align two point sets collected from left ventricle and lung surfaces. A look-up matrix was introduced to timprove the performance of the original ICP. The proposed CICP algorithm performs better than the original ICP (OICP) and Picky ICP (PICP) methods in not only the precision of the point correspondence estimation but also the resistance to noise and outliers. TPS-RPM was employed for matching cavity surface points to facilitate the dose accumulation among the high-dose-rate treatment fractions [35].

3D reconstruction is a vital procedure in medical image guided surgery where CT and MRI are both taken as pieces of slices initially. There will be dislocations between every two layers of the sequence of digital image as CT/MR slices due to the patient’s breathing, heart beating, and other physiological activities. Consequently, the first step of the segmentation and 3D reconstruction, which is based on the slices, is to register the upper and lower adjacent slice images. Although the active contour and active shape methods [46, 88, 89] have been proposed for the registration of slice images, the two main drawbacks of slice registration have not been conquered fundamentally. One drawback is that the quality of the initial position directly affects the registration result and speed, and the other is that the robustness and precision is rapidly reduced when there are inter-subject anatomical variabilities. To overcome these two drawbacks, Zhang et al. [90] proposed the KFM-PDM algorithm to align two point sets extracted from SIFT feature points of two neighboring MRI slices. A global deformable geometric model is obtained based on the point distribution model (PDM) of the obtained point sets. The Key Features Model (KFM), such as highly repeatable and highly robust SIFT features, was employed to adjust the initialized transformation parameters. The method has been freed from the harsh demand for the initial position of registration and improves its robustness while the inter-subject anatomical variabilities appear. However, manually extracted feature points restricted the application of PDM in other clinical scenes. CPD algorithm is a robust and efficient method for point set registration and has been used widely in the medical image registration. Peng et al. improved the CPD algorithm by using the shape context (SC-CPD) algorithm [5154] to find the correspondence between the point sets and to calculate the outlier ratio automatically to avoid manual assignments [71]. Point sets are extracted from the edges of slices to incorporate more information for registration. Although the registration accuracy is improved in the algorithm, the rising computational complexity cannot satisfy the real-time requirement of clinical applications. As an emerging algorithm, the applications of the CPD approach in medical image registration still have room for improvement.

4 Conclusions

  1. (1)

    In this paper, we focus on point-feature-based medical image registration which is extensively used in clinical applications. This paper is structured around the two core aspects: registration techniques and clinical application. For each component, particular emphasis is placed on classifying the methods according to their theoretical foundations and applications.

     
  2. (2)

    Independent of extrinsic points or intrinsic points, the precisely point acquisition is of great importance in the pre-procedure of the PMs. The accuracy of registration is affected severely by the precision of point extraction, therefore future work should explore more accurate feature segmented algorithms and points extraction methods. In addition, with the rapid development of the PMs, more precise and effective algorithms will be introduced into medical image registration.

     
  3. (3)

    Choosing proper geometric transformation methods is crucial after the feature points are acquired. If the correspondence between two point sets is known before registration, the least square method is sufficient to minimize the distance between two point sets. However, in most cases, the correspondence is unknown. Under these circumstances, parameters, such as the degree of deformation, the registration accuracy, and the signal-to-noise ratio, should be taken into consideration to choose an elegant geometric transformation. The RPM, GMM, and CPD are effective for non-rigid registration with large deformations, while ICP performs well at rigid and rough registration with low computational complexity. If considerable noises are consisted among the extracted point sets, algorithms with a low sensitivity to noise such as CPD should be chosen.

     
  4. (4)

    Clinical application is a new classification criterion that has been seldom related in the previous studies. As shown in Table 2 and Table 3, vascular interventional surgery is the most widely used field of the PMs. The main reason is that the centerlines, bifurcations and edge points of blood vessels can be used directly as the source of the feature points. Vascular interventional surgery of different vessels has different standards in the acquisition of feature points. For abdominal aortic vasculature, the vasculature diameter is very large, and centerline cannot describe the vessel structure precisely, so the edges of vessels are usually extracted as the source of the feature points. For liver vessels and vessels of other organs, diameters of these vessels are narrow and have limited influence on the registration accuracy, so the centerline of the vessel can directly serve as the origin of feature point sets.

     
  5. (5)

    In summary, feature-based medical image registration has greatly benefitted from its real-time performance and registration accuracy. With the rapid development of feature extraction technology, feature-based methods will experience an upcoming explosion.

     

Declarations

Authors’ Contributions

Cai Meng was directed the entire trial; Shao-Ya Guan wrote the manuscript; Tian-Miao Wang and Jun-Chen Wang assisted in modifying the structure and content of this paper. All authors read and approved the final manuscript.

Authors’ Information

Shao-Ya Guan, born in 1992, is currently a PhD candidate at School of Mechanical Engineering and Automation, Beihang University, China. She received her bachelor’s degree from Hebei University of Technology, China, in 2014. Her research interests include medical robotics and medical image processing.

Tian-Miao Wang, born in 1960, is currently a professor at School of Mechanical Engineering and Automation, Beihang University, China. He received his doctor’s degree from Northwestern Polytechnical University, China, in 1990. His research interests include medical robotics, special robots and service robots.

Cai Meng, born in 1977, is currently an associate professor at School of Astronautics, Beihang University, China. He received his doctor’s degree from Beihang University, China, in 2004. His research interests include machine vision and its application in robotic intelligent systems.

Jun-Chen Wang, born in 1984, is currently an associate professor at School of Mechanical Engineering and Automation, Beihang University, China. He received his doctor’s degree from Beihang University, China, in 2012. His research interests include medical robotics, medical image processing and minimally invasive surgery navigation.

Competing Interests

The authors declare no competing financial interests.

Funding

Supported by the National Natural Science Foundation of China (Grant No. 61533016)

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
School of Mechanical Engineering and Automation, Beihang University, Beijing, 100191, China
(2)
Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing, 100083, China
(3)
School of Astronautics, Beihang University, Beijing, 100191, China

References

  1. P Markelj, D Tomaževič, B Likar, et al. A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis, 2012, 16(3): 642–661.View ArticleGoogle Scholar
  2. S Matl, R Brosig, M Baust, et al. Vascular image registration techniques: A living review. Medical Image Analysis, 2017, 35: 1–17.View ArticleGoogle Scholar
  3. B Maiseli, Y Gu, H Gao. Recent developments and trends in point set registration methods. Journal of Visual Communication and Image Representation, 2017, 46: 95–106.View ArticleGoogle Scholar
  4. P J Besl, N D McKay. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239–256.View ArticleGoogle Scholar
  5. F P Oliveira, J M Tavares. Medical image registration: a review. Comput. Methods Biomech. Biomed Engin., 2014, 17(2): 73-93.View ArticleGoogle Scholar
  6. A Sotiras, C Davatzikos, N Paragios. Deformable medical image registration: A Survey. IEEE Transactions on Medical Imaging, 2013, 32(7): 1153–1190.View ArticleGoogle Scholar
  7. A P James, B V Dasarathy. Medical image fusion: A survey of the state of the art. Information Fusion, 2014, 19: 4–19.View ArticleGoogle Scholar
  8. T S Y Tang, R E Ellis, G Fichtinger. Fiducial registration from a single X-Ray image: a new technique for fluoroscopic guidance and radiotherapy. International Conference on Medical Image Computing and Computer-Assisted Intervention, Pittsburgh, USA, October 11–14, 2000: 502–511.Google Scholar
  9. A Schweikard, H Shiomi, J Adler. Respiration tracking in radiosurgery without fiducials. The International Journal of Medical Robotics and Computer Assisted Surgery, 2005, 1(2): 19–27.View ArticleGoogle Scholar
  10. R I Whyte. Stereotactic radiosurgery for lung tumors. Seminars in Thoracic and Cardiovascular Surgery. 2010, 22(1): 59–66.View ArticleGoogle Scholar
  11. Z Mu, D Fu, G Kuduvalli. A probabilistic framework based on hidden Markov model for fiducial identification in image-guided radiation treatments. IEEE Transactions on Medical Imaging, 2008, 27(9): 1288–1300.View ArticleGoogle Scholar
  12. R Balachandran, R F Labadie, J M Fitzpatrick. Clinical determination of target registration error of an image-guided otologic surgical system using patients with bone-anchored hearing aids. SPIE, California, United States, February 25-March 2, 2007, 6509: 650930.Google Scholar
  13. S D Olabarriaga, M Breeuwer, W J Niessen. Minimum cost path algorithm for coronary artery central axis tracking in CT images. International Conference on Medical Image Computing and Computer-Assisted Intervention, Montreal, Canada, November 15–18, 2003: 687–694.Google Scholar
  14. A F Frangi, W J Niessen, K L Vincken, et al. Multiscale vessel enhancement filtering. International Conference on Medical Image Computing and Computer-Assisted Intervention, Boston, USA, July 27–30, 1998: 130–137.Google Scholar
  15. M Schaap, T Walsum, L Neefjes, et al. Robust shape regression for supervised vessel segmentation and its application to coronary segmentation in CTA. IEEE Transactions on Medical Imaging. 2011, 30(11): 1974–1986.View ArticleGoogle Scholar
  16. M A Gülsün, H Tek. Robust vessel tree modeling. International Conference on Medical Image Computing and Computer-Assisted Intervention, New York, USA, September 6–10, 2008: 602–611.Google Scholar
  17. N Baka, C T Metz, C Schultz, et al. Statistical coronary motion models for 2D+t/3D registration of X-ray coronary angiography and CTA. Medical Image Analysis, 2013, 17(6): 698–709.View ArticleGoogle Scholar
  18. C T Metz, M Schaap, S Klein, et al. Registration of 3D+t coronary CTA and monoplane 2D+t X-ray angiography. IEEE Transactions on Medical Imaging, 2013, 32(5): 919–931.View ArticleGoogle Scholar
  19. D Ruijters, B M Romeny, P Suetens. Vesselness-based 2D-3D registration of the coronary arteries. International Journal of Computer Assisted Radiology and Surgery, 2009, 4(4): 391–397.View ArticleGoogle Scholar
  20. D Rivest-Henault, H Sundar, M Cheriet. Non-rigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions. IEEE Transactions on Medical Imaging, 2012, 31(8): 1557–1572.View ArticleGoogle Scholar
  21. J Kim, J Lee, W C Jin, et al. Locally adaptive 2D–3D registration using vascular structure model for liver catheterization. Computers in Biology and Medicine, 2016, 70: 119–130.View ArticleGoogle Scholar
  22. R Liao, Y Tan, H D Sun, et al. An efficient graph-based deformable 2D/3D registration algorithm with applications for abdominal aortic aneurysm interventions. International Workshop on Medical Imaging and Virtual Reality, Beijing, China, September 19–20, 2010: 561–570.Google Scholar
  23. H Lombaert, Y Sun, L Grady, et al. A multilevel banded graph cuts method for fast image segmentation. Tenth IEEE International Conference on Computer Vision (ICCV), Beijing, China, October 17–20, 2005: 259–265.Google Scholar
  24. K Palágyi, E Balogh, A Kuba, et al. A sequential 3D thinning algorithm and its medical applications. Biennial International Conference on Information Processing in Medical Imaging, Davis, USA, June 18–22, 2001: 409–415.Google Scholar
  25. S Habert, P Khurd, C H Chefd. Registration of multiple temporally related point sets using a novel variant of the coherent point drift algorithm: application to coronary tree matching. Medical Imaging: Image Processing, Florida, USA, February 10–12, 2013: 86690M.Google Scholar
  26. C V Stewart, C L Tsai, B Roysam. The dual-bootstrap iterative closest point algorithm with application to retinal mage registration. IEEE Transactions on Medical Imaging, 2003, 22(11): 1379–1394.View ArticleGoogle Scholar
  27. D Wassermann, J Ross, G Washko, et al. Deformable registration of feature-endowed point sets based on tensor fields. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA, June 24–27, 2014: 2729–2735.Google Scholar
  28. K Deng, J Tian, J Zheng, et al. Retinal fundus image registration via vascular structure graph matching. International Journal of Biomedical Imaging, 2010, 2010: 1–13.View ArticleGoogle Scholar
  29. M Groher, T F Jakobs, N Padoy, et al. Planning and intraoperative visualization of liver catheterizations. Academic Radiology, 2007, 14(11): 1325–1340.View ArticleGoogle Scholar
  30. C Liu, J Ma, Y Ma, et al. Retinal image registration via feature-guided Gaussian mixture model. Journal of the Optical Society of America A, 2016, 37(7): 1267–1276.View ArticleGoogle Scholar
  31. G Wang, Z Wang, Y Chen, et al. Robust point matching method for multimodal retinal image registration. Biomedical Signal Processing and Control, 2015, 19: 68–76.View ArticleGoogle Scholar
  32. J Chen, J Tian, N Lee, et al. A partial intensity invariant feature descriptor for multimodal retinal image registration. IEEE Transactions on Biomedical Engineering, 2010, 57(7): 1707–1718.View ArticleGoogle Scholar
  33. G Yang, C V Stewart, M Sofka, et al. Registration of challenging image pairs: Initialization, estimation, and decision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(11): 1973–1989.View ArticleGoogle Scholar
  34. M Fleute, S Lavallée. Nonrigid 3-D/2-D registration of images using statistical models. International Conference on Medical Image Computing and Computer-Assisted Intervention, Cambridge, UK, September 19–22, 1999: 138–147.Google Scholar
  35. X Zhen, H Chen, H Yan, et al. A segmentation and point-matching enhanced efficient deformable image registration method for dose accumulation between HDR CT images. Physics in Medicine and Biology, 2015, 60(7): 2981–3002.View ArticleGoogle Scholar
  36. G Zheng, M Á G Ballester, M Styner, et al. Reconstruction of patient-specific 3D bone surface from 2D calibrated fluoroscopic images and point distribution model. International Conference on Medical Image Computing and Computer-Assisted Intervention, Copenhagen, Denmark, October 1–6, 2006: 25–32.Google Scholar
  37. A Torsello, E Rodola, A Albarelli. Sampling relevant points for surface registration. International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), Hangzhou, China, May 16–19, 2011: 290–295.Google Scholar
  38. S D Billings, E M Boctor, R H Taylor. Iterative most-likely point registration (IMLP): A robust algorithm for computing optimal shape alignment, PLOS ONE, 2015, 10(3): e117688.View ArticleGoogle Scholar
  39. D C Barratt, G Penney, C S Chan, et al. Self-calibrating ultrasound-to-CT bone registration. International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI), Palm Springs, USA, October 26–29, 2005: 605–612.Google Scholar
  40. A Brounstein, I Hacihaliloglu, P Guy, et al. Towards real-time 3D US to CT bone image registration using phase and curvature feature based GMM matching. Medical Image Computing and Computer-Assisted Intervention(MICCAI), Toronto, Canada, September 18–22, 2011: 235–242.Google Scholar
  41. I Hacihaliloglu, B A Rounstein, P Guy, et al. 3D ultrasound-CT registration in orthopaedic trauma using GMM registration with optimized particle simulation-based data reduction. Medical Image Computing and Computer-Assisted Intervention(MICCAI), Nice, France, October 1–5, 2012: 82–89.Google Scholar
  42. G Penney, P Edwards, A King, et al. A stochastic iterative closest point algorithm (stochastICP). Medical Image Computing and Computer-Assisted Intervention(MICCAI), Utrecht, Netherlands, October 14–17, 2001: 762–769.Google Scholar
  43. S Granger, X Pennec. Multi-scale EM-ICP: A fast and robust approach for surface registration. European Conference on Computer Vision(ECCV), Copenhagen, Denmark, Jan 1–4, 2002: 418–432.Google Scholar
  44. A W Fitzgibbon. Robust registration of 2D and 3D point sets. Image and Vision Computing, 2003, 21(13): 1145–1153.View ArticleGoogle Scholar
  45. J J Moré. The Levenberg-Marquardt algorithm: implementation and theory. Numerical Analysis, the University of Wisconsin–Madison, USA, May 22–24, 1978: 105–116.Google Scholar
  46. J Yang, H Li, D Campbell, et al. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 2016, 38(11): 2241–2254.Google Scholar
  47. R A Mclaughlin, J Hipwell, D J Hawkes, et al. A comparison of a similarity-based and a feature-based 2-D-3-D registration method for neurointerventional use. IEEE Transactions on Medical Imaging, 2005, 24(8): 1058–1066.View ArticleGoogle Scholar
  48. S Gold, C P Lu, A Rangarajan, et al. New algorithms for 2D and 3D point matching: Pose estimation and correspondence. Advances in Neural Information Processing Systems, Denver, USA, November 27–30, 1995: 957–964.Google Scholar
  49. Rangarajan A, Chui H, Mjolsness E, et al. A robust point-matching algorithm for autoradiograph alignment. Medical Image Analysis, 1997, 1(4): 379–398.View ArticleGoogle Scholar
  50. Y Zheng, D Doermann. Robust point matching for nonrigid shapes by preserving local neighborhood structures. IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 2006, 28(4): 643–649.Google Scholar
  51. S Belongie, J Malik, J Puzicha. Shape context: A new descriptor for shape matching and object recognition. Advances in Neural Information Processing Systems, Vancouver, Canada, December 3–8, 2001: 831–837.Google Scholar
  52. S J Belongie, J Malik, J Puzicha. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509–522.View ArticleGoogle Scholar
  53. M Urschler, J Bauer, H Ditt, et al. SIFT and shape context for feature-based nonlinear registration of thoracic CT images. Computer Vision Approaches to Medical Image Analysis(CVAMIA), 2006, 6: 73–84.Google Scholar
  54. L Huang, Z Li. Feature-based image registration using the shape context. International Journal of Remote Sensing, 2010, 31(8): 2169–2177.View ArticleGoogle Scholar
  55. M Leordeanu, M Hebert. A spectral technique for correspondence problems using pairwise constraints. Tenth IEEE International Conference on Computer Vision(ICCV), Beijing, China, October 17–20, 2005: 1482–1489.Google Scholar
  56. H Liu, S Yan. Common visual pattern discovery via spatially coherent correspondences. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, USA, June 13–18, 2010: 1609–1616.Google Scholar
  57. L Torresani, V Kolmogorov, C Rother. Feature correspondence via graph matching: Models and global optimization. European Conference on Computer Vision(ECCV), Marseille, France, October 12–18, 2008: 596–609.Google Scholar
  58. H Liu, S Yan. Robust graph mode seeking by graph shift. 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, June 21–24, 2010: 671–678.Google Scholar
  59. S D Sousa, W G Kropatsch. Graph-based point drift: Graph centrality on the registration of point-sets. Pattern Recognition, 2015, 48(2): 368–379.View ArticleMATHGoogle Scholar
  60. S P Timoshenko, S K Woinowsky. Theory of plates and shells. McGraw-Hill, 1959.Google Scholar
  61. M Groher, D Zikic, N Navab. Deformable 2D-3D registration of vascular structures in a one view scenario. IEEE Transactions on Medical Imaging, 2009, 28(6): 847–860.View ArticleGoogle Scholar
  62. P E Gill, W Murray, M H Wright. Practical optimization. London: Academic Press, 1981.MATHGoogle Scholar
  63. B Jian, B C Vemuri. Robust point set registration using Gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1633–1645.View ArticleGoogle Scholar
  64. A P Dempster, N M Laird, D B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B (Methodological), 1977: 1–38.Google Scholar
  65. M S Yang, C Y Lai, C Y Lin. A robust EM clustering algorithm for Gaussian mixture models. Pattern Recognition, 2012, 45(11): 3950–3961.View ArticleMATHGoogle Scholar
  66. J Fan, J Yang, D Ai, et al. Convex hull indexed Gaussian mixture model (CH-GMM) for 3D point set registration. Pattern Recognition, 2016, 59: 126–141.View ArticleGoogle Scholar
  67. D Campbell, L Petersson. An adaptive data representation for robust point-set registration and merging. IEEE International Conference on Computer Vision(ICCV), Santiago, Chile, December 7–13, 2015: 4292–4300.Google Scholar
  68. N Baka, C T Metz, C J Schultz, et al. Oriented Gaussian mixture models for non-rigid 2D/3D coronary artery registration. IEEE Transactions on Medical Imaging, 2014, 33(5): 1023–1034.View ArticleGoogle Scholar
  69. A Myronenko, X Song, M A Carreira-Perpinán. Non-rigid point set registration: Coherent point drift. Advances in Neural Information Processing Systems, British Columbia, Canada, September 20–23, 2007: 1009–1016.Google Scholar
  70. A Myronenko, X Song. Point set registration: coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 2010, 32(12): 2262–2275.Google Scholar
  71. L Peng, G Li, M Xiao, et al. Robust CPD algorithm for non-rigid point set registration based on structure information. PloS One, 2016, 11(2): e148483.Google Scholar
  72. J N Dupej, V C Kraj, N J Pelik. Low-rank matrix approximations for Coherent point drift. Pattern Recognition Letters, 2015, 52: 53–58.View ArticleGoogle Scholar
  73. M Lu, J Zhao, Y Guo, et al. Accelerated coherent point drift for automatic three-dimensional point cloud registration. IEEE Geoscience and Remote Sensing Letters, 2016, 13(2): 162–166.View ArticleGoogle Scholar
  74. M Lu, J Zhao, Y Guo, et al. A 3D point cloud registration algorithm based on fast coherent point drift. IEEE Applied Imagery Pattern Recognition Workshop (AIPR), Washington, USA, October 14–16, 2014: 1–6.Google Scholar
  75. J Zhang, L Lian, J Lei, et al. Enhanced coherent point drift algorithm for remote sensing image registration. Journal of Applied Remote Sensing, 2015, 9(1): 095074.View ArticleGoogle Scholar
  76. W H Press. Numerical recipes 3rd edition: The art of scientific computing. London: Cambridge university press, 2007.MATHGoogle Scholar
  77. Y Khoo, A Kapoor. Non-iterative rigid 2D/3D point-set registration using semidefinite programming. IEEE Transactions on Image Processing, 2016, 25(7): 2956–2970.MathSciNetView ArticleGoogle Scholar
  78. A L Almhdie, C Ger, M Deriche, et al. 3D registration using a new implementation of the ICP algorithm based on a comprehensive lookup matrix: Application to medical imaging. Pattern Recognition Letters, 2007, 28(12): 1523–1533.View ArticleGoogle Scholar
  79. P Jannin, E Krupinski, S Warfield. Validation in medical image processing. IEEE Transactions on Medical Imaging, 2006, 25(11): 1405–1409.View ArticleGoogle Scholar
  80. P Felzenszwalb, D Huttenlocher. Distance transforms of sampled functions. New York: Cornell University, 2004.MATHGoogle Scholar
  81. Benincasa A B, Clements L W, Herrell S D, et al. Assessment of required intraoperative surface for accurate physical to image space registrations. Medical Physics, 2008, 35(9): 4251–4261.View ArticleGoogle Scholar
  82. Cash D M, Miga M I, Glasgow S C, et al. Concepts and preliminary data toward the realization of image-guided liver surgery. Journal of Gastrointestinal Surgery, 2007, 11(7): 844–859.View ArticleGoogle Scholar
  83. dos Santos T R, Seitel A, Kilgus T, et al. Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Medical Image Analysis, 2014, 18(7): 1101–1114.View ArticleGoogle Scholar
  84. Schmalz C, Forster F, Schick A, et al. An endoscopic 3D scanner based on structured light. Medical Image Analysis, 2012, 16(5): 1063–1072.View ArticleGoogle Scholar
  85. C Meng, J Zhang, D Liu, et al. A remote-controlled vascular interventional robot: system structure and image guidance. International Journal of Medical Robotics & Computer Assisted Surgery, 2013, 9(2): 230–239.View ArticleGoogle Scholar
  86. Maier-Hein L, Mountney P, Bartoli A, et al. Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Medical Image Analysis, 2013, 17(8): 974–996.View ArticleGoogle Scholar
  87. F GAO, Y ZHOU, F R DU, et al. Algorithm of pretreatment on automobile body point cloud. Chinese Journal of Mechanical Engineering, 2007, 20(4): 71–74.View ArticleGoogle Scholar
  88. S Rusinkiewicz, M Levoy. Efficient variants of the ICP algorithm. IEEE Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, Canada, May 28- June 1, 2001: 145–152.Google Scholar
  89. K Zhang, X Z Li, J X Zhang. A robust point-matching algorithm for remote sensing image registration. IEEE Geoscience and Remote Sensing Letters, 2014, 11(2): 469–473.View ArticleGoogle Scholar
  90. J Zhang, L Chen, X Wang, et al. Compounding local invariant features and global deformable geometry for medical image registration. PloS One, 2014, 9(8): e105815.View ArticleGoogle Scholar

Copyright

Advertisement