Optics-guided Robotic System for Dental Implant Surgery

Abstract

At present, dental implant surgery mainly relies on the clinical experience of the doctor and the assistance of preoperative medical imaging. However, there are some problems in dental implant surgery, such as narrow space, sight obstruction, inaccurate positioning, and high requirements of doctors' proficiency. Therefore, a dental implant robot system (DIRS) guided by optical navigation is developed in this study, with an x-shaped tool and an irregular pentagonal tracer are designed for spatial registration and needle tip positioning strategy respectively. The coordinate system of each unit in DIRS is unified through system calibration, spatial registration, and needle tip positioning strategy. Then the surgical path is planned on the computed tomography (CT) images in the navigation software before operation. The automatic positioning method and the auxiliary positioning method can be used in the operation to achieve accurate positioning and assist doctors to complete the operation. The errors of spatial registration, needle tip positioning strategy, and the overall accuracy of the system were evaluated respectively, and the results showed that they all met the needs of clinical surgery. This study preliminarily verified the feasibility of the precise positioning method for dental surgery robots and provided certain ideas for subsequent related research.

1 Introduction

With the deepening of aging, a lot of people are facing the situation of dental defect and loss, which is not only affect the appearance but also may cause severe pain [1]. A larger population is undertaking dental implants, both for treating diseases and for chewing food more conveniently [2]. The dental implant is the process of teeth root replacement where a strong foundation is provided. The emergence of dental implant surgery fills the traditional limitations of repair technique and becomes mature [3]. At present, dental implant surgery mainly depends on the doctor's experience and the assistance of computed tomography (CT) image [4], and there are some problems with the operation, such as occlusion of vision and narrow operation space [5]. Meanwhile, no matter how precise the plan is made in advance, the subjective intuition of the doctor still plays a decisive role in the operation. The exact location of implantation place has to be detected by experts a process that consumes a long time [6]. The planting nail deflection not merely affects the precision, and may even damage the maxillary nerve and mandibular nerve and cause unnecessary complications. A minimum of 2 mm around the implant is required for support [7]. Therefore, it is necessary to seek more convenient and accurate surgical methods for the dental implant. Surgical robots have high accuracy and can meet millimeter-level surgical requirements.

As a typical representative of mechatronics, surgical robot catalyzed the advance of minimally invasive surgery by incorporating the real-time stereoscopic vision of the operative field, eliminating the detrimental effect of hand or instrument tremors, and allowing for instrumentation with freedom of movement and precision beyond that of a surgeon’s hand and wrist [8,9,10]. Up to now, it has been widely used in orthopedic surgery and neurosurgery. For example, the ROSA Brain designed by Medtech is one of the most representative neurosurgical robots [11], and a new orthopedic surgical robot system called Tianji was listed by Tinavi recently [12]. However, the systems of those types have a large operating space and more acceptable. In the field of dental implant surgery [13], due to precision limitations, surgical robot technology is still not mature enough.

As early as 2010, Sun et al. [14] proposed the automated dental implantation using image-guided robotics, and completed the registration of surgical space and image space. The final registration result was (1.42 ± 0.7) mm. This work provides ideas for the following research but does not involve final positioning results. Li et al. [15] proposed a compact robotic system. The robot design utilized tendon-sheath transmission, by which the actuators could be placed in robot base, resulting a compact size and lightweight. The first robotic surgery system called ‘Yomi’ in the United States is approved for dental implant surgery in 2017 by U. S. Food and Drug Administration (FDA). This first system provides software for planning and navigational guidance for instrumentation during implant surgery [16]. The system also delivers haptic feedback and controls the position, depth, and angulation for implant osteotomy. The operational arm of the Yomi system assists in precise and automatic positioning by connecting a coordinate measurement machine (CMM) arm to the patient’s teeth. The operations of drilling and implant placement are still performed manually by the dentists using the operational arm. The operational arm itself does not perform any automated operations, and the CMM arm occupies the narrow operation space in the patients’ oral cavity [12]. So it is necessary to design a simpler registration tool to register image space and surgical space. Besides, a fully automatic dental implant system was developed by Yiming Zhao from the Air Force Medical University of China. This intelligent robot with a high degree of autonomy can automatically adjust continuously during intraoperative procedures, and can execute surgical tasks directly on patients without any apparent control by a surgeon [17]. However, limited confirming research is available regarding the reliability and feasibility of this system in clinical practice. So the robot must first be accurate positioning, to truly achieve a more safe and reliable dental implant robot clinical application.

Optical positioning is one of the common positioning methods. It has high accuracy and wide field of view (FoV) [18]. At the same time, it is not easy to be disturbed, and can provide accurate surgical navigation for the robot. At the same time, the coordinates of marker points recorded by optical positioning can also be used for spatial registration. Currently, the research about optical-based robot systems is divided into two categories consisting of the “eye in hand” type (EIH) and “eye to hand” type (ETH) [19, 20]. It is necessary to change the surgical instrument mounted on the end of the robot that may bring errors into the robot system. In addition, due to the oral cavity is narrow, the marker may be outside of the camera's FoV when the robot arm moves to a certain position [21]. In the robot system with ETH type, the marker and surgical instrument can be totally observed at the camera's FoV, and it is more flexible, more stable.

In this study, an optical-based dental implant robot system (DIRS), which adopts ETH type, is designed to assist dental implant surgery. An optical tracking system (OTS) provides the location information of retro-reflective markers mounted on the surgical tool and skull model. In order to realize hand-eye cooperation, a multiple closed-loop calibration (MCLC) algorithm is used to calculate the rotation and translation relationship between the robot and the OTS. At the same time, in order to achieve the precise positioning of the needle tip in the robot base coordinate system and the matching of the head model in the image space and the actual surgical space, an x-shaped tool and an irregular pentagonal tracer are designed respectively. Then the surgical instrument and model in image space and real space are matched by spatial registration, spatial position information of drill needle in robot coordinate system is obtained by needle tip positioning strategy. Finally, the appropriate surgical path and target can be selected according to the automatic positioning and auxiliary positioning methods, and the robot is controlled to reach the specified position along the path and perform the operation.

2 System Component

In Figure 1, the DIRS is composed of the following components: a six-degree-of-freedom (6 DoF) robotic arm (UR5e, Universal Robot Corporation) for actuating the surgical instrument; an OTS (AimPosition, Aimooe Corporation) for tracking the retro-reflective markers within the FoV; a self-designed x-shaped tool with four retro-reflective markers for needle tip positioning strategy and irregular pentagonal tracer with five retro-reflective markers for spatial registration; a navigation system integrated into a robot control center, which is Lenovo ThinkStation P720 with an Intel Xeon Silver 4210R at a 2.40 GHz CPU and a 32 GB RAM for image segmentation, algorithm analysis and robot communication control in software; a skull model and its CT image data is used as an experimental subject and a dental implant machine for drilling holes in the jaw. The robot control center can communicate with the robot and OTS through TCP/IP and USB respectively.

Figure 1

Illusion of the dental implant robot system

3 Methods

Robot movement can only be carried out in the robot base coordinate system, while the visual information is captured by OTS. Therefore, the coordinate system of each unit in the system should be unified through the following system calibration, space registration and needle tip positioning strategy.

3.1 System Calibration

System calibration of the robot and OTS is the foundation to surgical robot application, coordinate transformation relation between robot base to OTS and robot end effector to the surgical tool could be calculated through calibration. In this study, on the basis of the MCLC algorithm proposed previously [22], the coordinate value matrix of the marker points recorded by OTS can be used as an input to realize the system calibration. The calculation process of MCLC is divided into three loops, as shown in Figure 2, the first loop consists of robot base, robot end-effector, and OTS, the second loop consists of robot end-effector, surgical tool, dental implant machine needle tip, and OTS, and the first two loops are connected to get the third loop. The following procedure obtains the input matrix required for the MCLC algorithm.

Figure 2

Diagram of multiple closed-loops calibration algorithm

$C_{x}$ is the coordinate system of $x$, ${\varvec{R}}_{xy}$ and ${\varvec{T}}_{xy}$ denote the rotation matrix and translation matrix from $C_{x}$ to $C_{y}$ respectively. The lowercase meaning of each coordinate system is shown in Table 1.

Table 1 The lowercase meaning

To begin with, a series of initial pose $p_{i}$ of robot end effector in $C_{b}$ should be selected, and $p_{i} = (x_{i} ,y_{i} ,z_{i} ,rx_{i} ,ry_{i} ,rz_{i} )$,$(x_{i} ,y_{i} ,z_{i} )$ and $(rx_{i} ,ry_{i} ,rz_{i} )$ stand for the position and orientation respectively. The robot starts from each initial position and moves to the offset position. The offset pose is calculated according to the initial pose and the axial direction of the base coordinate system $C_{b}$ and the end effector coordinate system $C_{e}$ is the extension direction.

In the process of the robot moving from the initial pose to all the offset positions, the reflective markers on the surgical tool as shown in Figure 3 should be located in the FoV of the OTS to ensure that the OTS can capture the markers in real time. In order to reduce the inherent error, this algorithm takes a total of 25 initial poses. After the robot reaches each initial pose, it is offset 4 times along each coordinate axis of $C_{b}$, and the offset distance is $\delta$ each time in Eq. (1). When the robot moves along the $C_{b}$ axis to each target pose, the spatial position of the three non-collinear reflective markers on the surgical tool is recorded as Eq. (2):

$$\left\{ {\begin{array}{*{20}l} {p_{i}^{jx} = \left( {x_{i} + j\delta ,y_{i} ,z_{i} ,rx_{i} ,ry_{i} ,rz_{i} } \right),} \hfill \\ {p_{i}^{jy} = \left( {x_{i} ,y_{i} + j\delta ,z_{i} ,rx_{i} ,ry_{i} ,rz_{i} } \right),} \hfill \\ {p_{i}^{jz} = \left( {x_{i} ,y_{i} ,z_{i} + j\delta ,rx_{i} ,ry_{i} ,rz_{i} } \right),} \hfill \\ \end{array} } \right.$$

(1)

$$\left( {\begin{array}{*{20}l} {_{b} P_{a}^{x} } \hfill & {_{b} P_{b}^{x} } \hfill & {_{b} P_{c}^{x} } \hfill \\ {_{b} P_{a}^{y} } \hfill & {_{b} P_{b}^{y} } \hfill & {_{b} P_{c}^{y} } \hfill \\ {_{b} P_{a}^{z} } \hfill & {_{b} P_{b}^{z} } \hfill & {_{b} P_{c}^{z} } \hfill \\ \end{array} } \right)_{3 \times 9} ,$$

(2)

where, $_{b} P_{k} = (x_{k} ,y_{k} ,z_{k} )$ represents three-dimensional coordinates, and the subscript $k \in (a,b,c)$ represents the serial number of the markers.

Figure 3

Surgical tool coordinate system established by three non-collinear retro-reflective markers

After completing the above motion along the base coordinate system $C_{b}$, the robot should perform the motion under the end-effector coordinate system $C_{e}$. The manipulator moves forward along the coordinate axes of the end coordinate system $C_{e}$ based on this pose. But the robot motion is carried out in the base coordinate system. Therefore, the target offset distance $\delta$ should be mapped from the end coordinate system $C_{e}$ to the base coordinate system $C_{b}$. Therefore, the rotation matrix ${\varvec{R}}_{be}$ is obtained by quaternion method [23] and rotation component $(rx_{i} ,ry_{i} ,rz_{i} )$ of initial pose. Hence, the ${{\varvec{\Delta}}}_{e}^{x} ,{{\varvec{\Delta}}}_{e}^{y} ,{{\varvec{\Delta}}}_{e}^{z}$, which represent the j-th offset distance value along $X,Y,Z$ axis of $C_{e}$ respectively, can be expressed as:

$$\left\{ {\begin{array}{*{20}l} {{{\varvec{\Delta}}}_{e}^{x} = j\delta \cdot {\varvec{R}}_{eb} \cdot {\varvec{e}}_{x} ,} \hfill \\ {{{\varvec{\Delta}}}_{e}^{y} = j\delta \cdot {\varvec{R}}_{eb} \cdot {\varvec{e}}_{y} ,} \hfill \\ {{{\varvec{\Delta}}}_{e}^{z} = j\delta \cdot {\varvec{R}}_{eb} \cdot {\varvec{e}}_{z} ,} \hfill \\ \end{array} } \right.$$

(3)

where, ${\varvec{e}}_{x} = (1,0,0)^{\text{T}}$,${\varvec{e}}_{y} = (0,1,0)^{\text{T}}$, ${\varvec{e}}_{z} = (0,0,1)^{\text{T}}$, and ${{\varvec{\Delta}}}_{e}^{x} = \left( {\delta_{e1}^{jx} ,\delta_{e2}^{jx} ,\delta_{e3}^{jx} } \right)^{\text{T}}$, ${{\varvec{\Delta}}}_{e}^{y} = \left( {\delta_{e1}^{jy} ,\delta_{e2}^{jy} ,\delta_{e3}^{jy} } \right)^{\text{T}}$,

${{\varvec{\Delta}}}_{e}^{z} = \left( {\delta_{e1}^{jz} ,\delta_{e2}^{jz} ,\delta_{e3}^{jz} } \right)^{\text{T}}$. Therefore, for different initial pose $p_{i}$, the target offset pose $p_{ie}^{jx} ,p_{ie}^{jy} ,p_{ie}^{jz}$ of the j-th order of the forward motion along the X, Y and Z axes of $C_{e}$ can be expressed as:

$$\left\{ {\begin{array}{*{20}l} {p_{i}^{jx} = \left( {x_{i} + \delta_{e1}^{jx} ,y_{i} + \delta_{e2}^{jx} ,z_{i} + \delta_{e3}^{jx} ,rx_{i} ,ry_{i} ,rz_{i} } \right),} \hfill \\ {p_{i}^{jy} = \left( {x_{i} + \delta_{e1}^{jy} ,y_{i} + \delta_{e2}^{jy} ,z_{i} + \delta_{e3}^{jy} ,rx_{i} ,ry_{i} ,rz_{i} } \right),} \hfill \\ {p_{i}^{{j_{z} }} = \left( {x_{i} + \delta_{e1}^{jz} ,y_{i} + \delta_{i2}^{jz} ,z_{i} + \delta_{e3}^{jz} ,rx_{i} ,ry_{i} ,rz_{i} } \right).} \hfill \\ \end{array} } \right.$$

(4)

Similarly, when the robot moves along the $C_{e}$ axis to the target offset position, the spatial position information of the three reflective markers can be recorded as:

$$\left( {\begin{array}{*{20}l} {_{e} P_{a}^{x} } \hfill & {_{e} P_{b}^{x} } \hfill & {_{e} P_{c}^{x} } \hfill \\ {_{e} P_{a}^{y} } \hfill & {_{e} P_{b}^{y} } \hfill & {_{e} P_{c}^{y} } \hfill \\ {_{e} P_{a}^{z} } \hfill & {_{e} P_{b}^{z} } \hfill & {_{e} P_{c}^{z} } \hfill \\ \end{array} } \right)_{3 \times 9} .$$

(5)

The value of ${\varvec{R}}_{bo} ,{\varvec{R}}_{et} ,{\varvec{T}}_{bo} ,{\varvec{T}}_{et}$ can be calculated by using Eqs. (2) and (5) as the input data of MCLC algorithm [22].

3.2 Spatial Registration

Matching of actual surgical space and image space is called spatial registration, the accuracy of spatial registration directly relates to the accuracy of DIRS. The bone-anchorage fiducial is considered the gold standard due to its rigid attachment and clear CT image contrasting the registration template and anatomic landmarks [24, 25]. In this study, we designed an irregular pentagonal retro-reflective marker tracer in Figure 4, which is made of resin materials by 3D-printing, and the function of the tracer is to fix 5 retro-reflective markers $p_{j}$.

Figure 4

Illusion of the irregular pentagonal retro-reflective marker tracer

Figure 5a shows a rigid structure can be formed by sticking tracer onto the teeth of a skull model. After CT scanning and 3D visualization reconstruction, a 3D skull model image can be obtained in the navigation software system of the robot control center as shown in Figure 5b. The Euclidean distance between every two markers on the 3D skull model can be calculated by image segment. After each column element of the Euclidean distance matrix is arranged in descending order, it can be expressed as:

$$_{i} {\varvec{D}} = \left[ {\begin{array}{*{20}l} {_{i} d_{1} } \hfill & {_{i} d_{2} } \hfill & {_{i} d_{3} } \hfill & {_{i} d_{4} } \hfill \\ \end{array} } \right].$$

(6)

Figure 5

a Skull model in actual surgical space, b Skull model in image space

When the skull model is placed in the FoV of OTS, markers on the tracer can be identified and tracked. And Euclidean distance of every two markers can be calculated and arranged in descending order similarly as:

$$_{s} {\varvec{D}} = \left[ {\begin{array}{*{20}l} {_{s} {\varvec{d}}_{1} } \hfill & {_{s} {\varvec{d}}_{2} } \hfill & {_{s} {\varvec{d}}_{3} } \hfill & {_{s} {\varvec{d}}_{4} } \hfill \\ \end{array} } \right],$$

(7)

$$_{s} {\varvec{d}}_{j} = \left[ {\left\| {p_{j} - p_{1} } \right\|, \cdots ,\left\| {p_{j} - p_{j - 1} } \right\|,\left\| {p_{j} - p_{j + 1} } \right\|, \cdots ,\left\| {p_{j} - p_{5} } \right\|} \right]^{\text{T}} ,$$

(8)

$$_{i} {\varvec{d}}_{j} = \left[ {\left\| {q_{j} - q_{1} } \right\|, \cdots ,\left\| {q_{j} - q_{j - 1} } \right\|,\left\| {q_{j} - q_{j + 1} } \right\|, \cdots ,\left\| {q_{j} - q_{5} } \right\|} \right]^{\text{T}} ,$$

(9)

where, matrix $_{i} {\varvec{D}}$ matrix $_{s} {\varvec{D}}$ are both 4×4 order matrices; $_{s} {\varvec{d}}_{j}$ and $_{i} {\varvec{d}}_{j}$ respectively represent the Euclidean distance between the j-th marker and the other four markers in the surgical space and image space. And then the elements in the matrix $_{s} {\varvec{D}}$ are subtracted from the corresponding elements of the matrix $_{i} {\varvec{D}}$ to obtain the error matrix ${\varvec{E}}$.

$${\varvec{E}} = \left[ {\begin{array}{*{20}l} {{\varvec{e}}_{1} } \hfill & {{\varvec{e}}_{2} } \hfill & {{\varvec{e}}_{3} } \hfill & {{\varvec{e}}_{4} } \hfill \\ \end{array} } \right],$$

(10)

$${\varvec{e}}_{j} = \left[ {\begin{array}{*{20}l} {e_{j1} } \hfill & {e_{j2} } \hfill & {e_{j3} } \hfill & {e_{j4} } \hfill \\ \end{array} } \right]^{\text{T}} .$$

(11)

If the minimum element of the first column vector ${\varvec{e}}_{1}$ in matrix ${\varvec{E}}$ is $e_{1k}$, then the marker $p_{1}$ in the surgical space corresponds to the marker $q_{k}$ in the image space. Then the quaternion method can be used to obtain the coordinate transformation relationship between the surgical space and the image space.

3.3 Needle Tip Positioning Strategy

As shown in Figure 3, three non-collinear retro-reflective markers construct the surgical tool coordinate system $C_{t}$. To implement precise positioning, it is necessary to calculate the coordinate value of the drill needle tip in $C_{t}$. OTS can track and locate retro-reflective markers in the FoV to establish surgical tool coordinate system $C_{t}$, while the dental implant machine's needle tip coordinate system $C_{p}$ can be established by markers similarly. The rotation matrix ${\varvec{R}}_{bo} ,{\varvec{R}}_{et} ,{\varvec{R}}_{be}$ and translation matrix ${\varvec{T}}_{bo} ,{\varvec{T}}_{et} ,{\varvec{T}}_{be}$ have been calculated in system calibration, but the relationship between $C_{t}$ and $C_{p}$ remains unknown. In clinical practice, it is necessary to replace drill needles in Figure 6(d) with different diameters and lengths during operation. Hence, a separate x-shaped tool is designed to establish $C_{p}$ and determine the position of the drill needle tip in that coordinate system.

Figure 6

The x-shaped tool: a Vertical view, b Establish the coordinate system of $C_{p}$, c Use the x-shaped tool to determine the position of the needle tip and keep it vertical to determine the direction of the needle body, d Some drill needles of different sizes

Figure 6a shows a hole of 3 mm in diameter and 3 mm in depth at the center of the x-shaped tool. This hole is used to determine the direction and tip position of the needle. As shown in Figure 6c, when using it to locate the tip of the needle, the drill needle needs to be vertical to the center hole of the x-shaped tool. Then let the center at the bottom of the hole be the origin of $C_{p}$, the ${\varvec{V}}_{x}$ pointing horizontally to $P_{m}$ be the $X$ axis, ${\varvec{V}}_{y}$ be the $Y$ axis. So the $Z$ axis ${\varvec{V}}_{z}$ shown in Figure 6b, can be expressed as

$${\varvec{V}}_{z} = {\varvec{V}}_{x} \times {\varvec{V}}_{y} .$$

(12)

After theoretical measurement and calculation, the rough spatial coordinates of the four markers' center can be obtained. At the same time, the origin and a point on the z-axis of the $C_{p}$ are selected as needle tip and needle body respectively. These two points define the orientation of the needle. To reduce errors, OTS is needed to calibrate them. The calibrated values are shown in the Table 2.

Table 2 The calibrated coordinate values.

In the preoperative phase, doctors choose the appropriate drill needle according to the clinical operation requirements and install it on the dental implant machine. After changing different drill needles, the position of the needle tip in $C_{t}$ is changed and needs to be corrected. Once corrected, the positional and orientational relationship between $C_{t}$ and $C_{p}$ is considered as a constant. Therefore, we can use singular value decomposition (SVD) [26] to analyze the point set $\left\{ {_{t} P_{m} ,_{t} P_{n} ,_{t} P_{o} ,_{t} P_{p} } \right\}$, $\left\{ {_{p} P_{m} ,_{p} P_{n} ,_{p} P_{o} ,_{p} P_{p} } \right\}$ in $C_{t}$, $C_{p}$ and get rotation matrix ${\varvec{R}}_{tp}$ and translation matrix ${\varvec{T}}_{tp}$.

The values in $\left\{ {_{p} P_{m} ,_{p} P_{n} ,_{p} P_{o} ,_{p} P_{p} } \right\}$ are available from Table 2, while the content in $\left\{ {_{t} P_{m} ,_{t} P_{n} ,_{t} P_{o} ,_{t} P_{p} } \right\}$ needs to be mapped through multiple coordinate systems. As shown in Figure 6c, hold the x-shaped tool vertically to the drill needle and ensure that the end of the drill needle is in the central hole of the x-shaped tool, which can determine not only the position of the tip, but also the direction of the needle. In the meantime, OTS locates those retro-reflective markers and records their coordinates in $C_{o}$, which can be represented as $_{o} P_{j}$ ($_{o} P_{j} = (x_{j} ,y_{j} ,z_{j} ),j = m,n,o,p$). Furthermore, the rotation matrices ${\varvec{R}}_{xy}$ and ${\varvec{R}}_{yx}$ are inverses to each other, the translation matrix ${\varvec{T}}_{yx}$ can be expressed as:

$${\varvec{T}}_{yx} = - {\varvec{R}}_{xy}^{ - 1} {\varvec{T}}_{xy} .$$

(13)

The rotation matrix ${\varvec{R}}_{bo} ,{\varvec{R}}_{et} ,{\varvec{R}}_{be}$ and translation matrix ${\varvec{T}}_{bo} ,{\varvec{T}}_{et} ,{\varvec{T}}_{be}$ have been calculated in system calibration. Then the points in $\left\{ {_{t} P_{m} ,_{t} P_{n} ,_{t} P_{o} ,_{t} P_{p} } \right\}$ can be mapped through multiple coordinate systems as follows:

$$\left[ {\begin{array}{*{20}c} {_{t} P_{j} } \\ 1 \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {{\varvec{R}}_{te} } & {{\varvec{T}}_{te} } \\ 0 & 1 \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {{\varvec{R}}_{eb} } & {{\varvec{T}}_{eb} } \\ 0 & 1 \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {{\varvec{R}}_{bo} } & {{\varvec{T}}_{bo} } \\ 0 & 1 \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {_{o} P_{j} } \\ 1 \\ \end{array} } \right].$$

(14)

Using SVD to analyze those two point set, the rotation and translation relationship between $C_{t}$ and $C_{p}$ can be calculated. In this way, the position of $C_{p}$'s origin, which is also the position of the needle tip, can be accurately determined after any drill needle is replaced.

3.4 Surgical Strategy

The transformation relationship of each coordinate system in DIRS can be obtained, after completing the above system calibration, spatial registration, and needle tip positioning strategy. The surgical path planning of CT image in surgical navigation software can be transmitted to the robot through socket communication to complete the positioning. Firstly, the CT image series is imported into the robot control center and divided into three sections including sagittal, coronal, and axial plane. In order to comprehensively examine the nerves, blood vessels, and cavities in the jaw region, three-dimensional visualization reconstruction of CT images is needed, which can also help doctors to design a more perfect preoperative plan. As shown in Figure 7, a 3D image of a skull model defined as $C_{i}$ is reconstructed and rendered by VTK (Visualization Toolkit). In spatial registration, the relationship between image space and surgical space has been elaborated. 3D image in $C_{i}$ can provide multiple functions such as omnidirectional inspection, transparency adjustment, and image zooming, which can be effectively used in preoperative surgical plan formulation.

Figure 7

3D image, axial, sagittal and coronal plane in $C_{i}$

A drill needle movement path can be plan in the image space, which includes safety points $P_{i}^{s}$ and target point $P_{i}^{t}$. Therefore, the safety points and target point should be mapped from the image space to the robot base coordinate system. The calculation process is as follows:

$$\left( {\begin{array}{*{20}c} {P_{b}^{s} } \\ 1 \\ \end{array} } \right) = \left( {\begin{array}{*{20}c} {{\varvec{R}}_{ob} } & {{\varvec{T}}_{ob} } \\ 0 & 1 \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {{\varvec{R}}_{io} } & {{\varvec{T}}_{io} } \\ 0 & 1 \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {P_{i}^{s} } \\ 1 \\ \end{array} } \right),$$

(15)

$$\left( {\begin{array}{*{20}c} {P_{b}^{t} } \\ 1 \\ \end{array} } \right) = \left( {\begin{array}{*{20}c} {{\varvec{R}}_{ob} } & {{\varvec{T}}_{ob} } \\ 0 & 1 \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {{\varvec{R}}_{io} } & {{\varvec{T}}_{io} } \\ 0 & 1 \\ \end{array} } \right)\left( {\begin{array}{*{20}c} {P_{i}^{t} } \\ 1 \\ \end{array} } \right).$$

(16)

In this study, we provide automatic and auxiliary positioning methods. The so-called automatic positioning method is to set the target point on the surface of alveolar bone in Figure 8a, accurately control the drill needle to reach the safety point first along the path, and then reach the surface of alveolar bone from the safety point at a slower speed to perform the drilling operation. The process is fully automated, and the robot's movement path is customized by the doctor before surgery according to the patient's condition. As shown in Figure 8b, the auxiliary positioning method refers to setting the target outside the alveolar bone. After the tip reaches the target, the UR5e robot switches to the force mode, and then the doctor can manually push the robot along the fixed path. Since the path selection under the force mode of UR5e is limited, the pose of the patient must be considered in advance before use. This method is combined by the robot and manual, which has higher safety.

Figure 8

Surgical path planning: a Automatic positioning method, set the target on the alveolar bone, b Auxiliary positioning method, set the target point outside the alveolar bone

4 Experiments and Results

As discussed earlier, due to dental implantation is the process of fixing Titanium mental pin into the jaw to perform the teeth restoration process, the accuracy of drill needle positioning directly determines the success or failure of robot-assisted surgery. In this study, the skull model was assumed to be static during the operation. The error sources of DIRS are mainly accumulated in three parts, namely the inherent error of the robot, the inherent error of the OTS, and the algorithm error. The errors of the first two are ±0.03 mm and ±0.12 mm respectively [27, 28], which meet the requirements of dental implant surgery. And due to the mechanical manufacturing process, these errors are usually unavoidable. Therefore, only the error of the algorithm needs to be considered in this system.

The robot is calibrated by the method described in the system calibration to obtain the rotation and translation relationship between the various components in DIRS. As shown in the Figure 9, 25 initial pose points within the OTS field of view are selected for system calibration. The calibration results are shown in the Table 3.

Figure 9

25 initial pose points for system calibration in $C_{b}$(The small blue, green and yellow squares represent the projection of $P_{i}$ on the plane)

Table 3 The results of system calibration.

When the calibration of the system is completed, the position and orientation of the robot and the skull model are adjusted to make both of them in the FoV of OTS for the next step. The first is spatial registration. The built-in infrared sensor in OTS receives the reflected infrared light from the markers to track the spatial position of the markers on the tracker in real-time. Matching the received spatial position information with the marker set in the image space to complete the spatial registration. The rotation and translation matrices of the skull model in different spaces can be obtained by the quaternion or the SVD. Therefore, we can convert the virtual point coordinate value in the navigation software of the robot control center to the point coordinate value in the actual surgical space to achieve path planning. A single spatial registration takes only about 0.5 ms. Usually, in order to reduce accidental errors, 200 spatial registrations are carried out during actual operation and the average error value is calculated. As shown in Figure 10, we have verified 8 sets of spatial registration values, the error is (0.19625±0.05263) mm. Next is the tip positioning strategy. Similarly, we transform the origin of the $C_{p}$ into a coordinate value in the $C_{t}$, which is the position of the tip. Then, eight groups of needle tip positioning error values were calculated, and each group was the average of 200 calculated values. As shown in Figure 10, the needle tip positioning error is (0.24875±0.04224) mm.

Figure 10

The error of spatial registration and needle tip positioning strategy

After the above preoperative preparation, the surgical path can be set on the navigation software in robot control center. As shown in Figure 8a, for automatic positioning method, the target is set on a ceramic bead with a diameter of 1 mm in Figure 11a that is embedded in the alveolar bone. With the planed path in $C_{i}$, a target pose in $C_{b}$ of robot could be computed by using the presented algorithm, and the precision is shown in Figure 11a. Through 8 repeated tests, the final result is as shown in the Figure 11c, and the mean precision error is (0.9375±0.3021) mm, which can satisfy the requirements of clinical dental implant surgery. The most fundamental difference between the auxiliary positioning method and the automatic positioning method is that the target point is different. The auxiliary positioning method requires more intervention by doctors to ensure that accidents are avoided. As shown in the Figure 12a, after the needle tip reaches the target point, the doctor uses the UR5e force mode to manually drive the needle tip to move along a fixed path. Figure 12b–d shows the needle tip trajectory, the angle of each joint and the angular velocity of the joint respectively when the auxiliary positioning method is adopted. But its path has more limitations and must be considered before surgery, and it takes more time.

Figure 11

The results of automatic positioning method: a Tip positioning result display, b Most of the eight final positioning results are within 1mm, which can meet the actual needs of dental implant surgery

Figure 12

The results of auxiliary positioning method: a After the needle tip reaches the target point, manually push the robotic arm to move along the fixed path in the UR5e force mode, b The trajectory of the needle tip, c Real-time angle of each joint of the robot, d Real-time angular velocity of each joint of the robot, the time period when the speed is zero represents the program interval stage

5 Discussion and Conclusions

The results of the experiment indicated that the robot could precisely actuate the drill needle tip toward the target point. The DIRS meet the demands of high accuracy of implant placement and positioning. Due to the limitation of patient's mouth opening and the location of loss, doctor usually need to perform long operations in restricted situations. Human error is more susceptible to physical strength and fatigue. In addition, the robot showed sustained accuracy and higher stability in previous repeated experiments. Nonetheless, robot-assisted dental implants are still in research worldwide, with only a few doctors using similar techniques. Schneider et al. reported that the mean deviation value of apex is 1.63 mm by using computer-aided implant positioning technology [29]. As a representative product of dental implant robot, YOMI system adopts the assisted arm positioning method, and its positioning deviation value is 0.8 mm [16]. Therefore, compared with the YOMI system, DIRS has similar positioning accuracy, but the oral space occupied by the tracer used for spatial registration is smaller and lighter, which will not hinder the robot. Both automatic positioning and auxiliary positioning methods have high accuracy, and the path is carefully designed by professional dentists before operation, and the robot will not exceed the preset path. In fact, the manipulator itself also has the function of emergency braking. In case of emergencies, doctors can directly intervene in braking to ensure safety.

The spatial registration and needle tip positioning strategy based on optical markers has achieved good results. Compared with the autonomous oral and maxillofacial surgery robot proposed by Ma et al. [30], which can markerlessly detect the pose of the skull model, its final positioning error is larger than 1mm. The spatial registration and needle-point positioning strategy we designed not only occupies a small space, but also has a higher accuracy of identification and tracking, and the accumulated error is smaller. Meanwhile, the time consuming each time is very short. It only takes 0.5 milliseconds for a single mark recognition calculation, shortening the time of robot-assisted surgery. The pentagonal tracer used for spatial registration needs to be customized according to the patient's teeth, and it must be closely embedded on the teeth during the actual registration process. If it becomes loose, it may cause positioning failure. Likewise, the needle point positioning strategy also has requirements. It is necessary to ensure that the drill needle is perpendicular to the x-shaped tool so that the robot can adjust the angle of the drill needle to the angle required by the implant before reaching the target point.

Overall, there are certain limitations in this study. Firstly, we did not carry out animal experiment verification, which is also the work we need to do in the future. Secondly, if the automatic positioning method is adopted, the patient should remain static in the operation, and the auxiliary positioning method can only guide the doctor to control the drill needle to move along a fixed path, which requires more intervention from the doctor. Finally, although the experimental results show that the positioning accuracy is less than 1mm, since the system itself is complex and has many sources of error, the accuracy cannot be guaranteed after the error of each unit is accumulated. This system still needs to reduce the source of system error and a large number of experimental verification to ensure the safety and feasibility of the system.

In summary, this paper developed a DIRS to assist dental implant surgery. The spatial position relationships of each unit in DIRS are unified by system calibration, spatial registration, and needle tip positioning strategy. The system can achieve high accuracy either by automatic positioning method or by auxiliary positioning method. With the deepening of automation and intelligence of medical devices, more intelligent medical instruments will appear in future clinical surgery. The purpose of this system is to establish a more standardized surgical procedure, design a better surgical plan in the surgical navigation software in pre-operation, and assist doctors in accurately positioning the target area for drilling. The advantage is that the surgical robot has good stability, and the preoperative scheme is consistent with the intraoperative positioning. This work may be able to provide new ideas and solutions for the development of robots in the field of dental implants.