An Adaptive Hand Exoskeleton for Teleoperation System

Abstract

Teleoperation can assist people to complete various complex tasks in inaccessible or high-risk environments, in which a wearable hand exoskeleton is one of the key devices. Adequate adaptability would be available to enable the master hand exoskeleton to capture the motion of human fingers and reproduce the contact force between the slave hand and its object. This paper presents a novel finger exoskeleton based on the cascading four-link closed-loop kinematic chain. Each finger has an independent closed-loop kinematic chain, and the angle sensors are used to obtain the finger motion including the flexion/extension and the adduction/abduction. The cable tension is changed by the servo motor to transmit the contact force to the fingers in real time. Based on the finger exoskeleton, an adaptive hand exoskeleton is consequently developed. In addition, the hand exoskeleton is tested in a master–slave system. The experiment results show that the adaptive hand exoskeleton can be worn without any mechanical constraints, and the slave hand can follow the motions of each human finger. The accuracy and the real-time capability of the force reproduction are validated. The proposed adaptive hand exoskeleton can be employed as the master hand to remotely control the humanoid five-fingered dexterous slave hand, thus, enabling the teleoperation system to complete complex dexterous manipulation tasks.

1 Introduction

Teleoperation fuses human knowledge and experience and provides support for human to remote manipulation. Thus, it is widely applied to surgical robots, drones, unmanned vehicles, and other fields [1]. For high-risk environments with complex manipulating scenarios, such as bomb demolition, dangerous goods disposal, oil, and gas valve operation, it is necessary to have a humanoid five-fingered dexterous slave hand with flexible operational capabilities such as screwing, pushing, pulling, and pressing [2]. Therefore, the master hand exoskeleton is expected to be worn by humans to acquire agile motion.

As one of the key devices of the master–slave teleoperation system, the master exoskeleton hand is required to map the motion of each finger to the bionic five-fingered dexterous slave hand and transmit the interaction force between the hand and its object to the human hand [3, 4]. In general, the following three problems should be considered in the design of the master hand exoskeleton. At first, the motion of fingers should not be mechanically constrained, even for different-sized hands. Secondly, there tends to be coupling motion between the joints of the human finger, which sets the challenge of accurately capturing the angle of each joint. Thirdly, it should reproduce the interactive force from the slave hand in real time.

Amirpour et al. [5, 6] proposed a two-fingered hand exoskeleton. Each finger has three degrees of freedom (DOF), and the end-to-end single-point connection forms a motion-closed chain, which can remotely operate the mechanical gripper to complete target grasping. Because the captured human motion is limited to the thumb and index finger, the exoskeleton cannot cover the entire hand. So, it is incapable of conducting the above-mentioned multi-task operation. Park et al. [7, 8] proposed a three-fingered hand exoskeleton WeHAPTIC for manipulating objects in virtual reality applications. The exoskeleton consisted of two rods connecting the fingertip and the opisthenar to form a closed-loop kinematic chain. Based on the fingertip posture estimation algorithm, the kinematic angle of the finger joint was solved. For different users, the estimation algorithm needed to be calibrated in advance. It undoubtedly increases the complexity of the measurement operation. The complex kinematic model also reduces the accuracy of motion capture. Jo et al. [9, 10] proposed a portable hand exoskeleton system to assist finger movements with just one DOF, which could only implement flexion and extension motion of the four fingers outside the thumb without taking into account adduction/abduction motion. The University of Pisa developed a hand exoskeleton ExoSense for tracking human hand operations [11]. Each finger housed a sensor to obtain the position of its object. The system also needed calibration before use. Furthermore, the exoskeleton only had a tactile detection function, unable to achieve force-sensing reproduction. The force feedback exoskeleton (Dexmo) developed by Dexta Company could feedback on the interacting force from the exoskeleton and its object through a motor with a high gear ratio [12, 13]. However, when the slave hand did not interact with the object, it was difficult to simulate the free movement of fingers at low impedance because of the large transmission ratio, which limited the performance of the human hand.

In addition to the above-mentioned commonly used direct current (DC) motor [14], pneumatic artificial muscle (PAM) antagonism [15], and magnetorheological fluid damper [16] are also employed as actuating techniques for the hand exoskeleton. Zheng et al. [17] proposed a low-impedance force feedback data glove based on PAM, which satisfied the low-impedance movement of the master hand in free space. Although PAM offers low stiffness, there is a delay in force reproduction.

To realize the dexterous operation of the humanoid five-fingered dexterous slave hand, this paper presents a novel closed chain cascade configuration of finger joints for the adaptive hand exoskeleton. Each finger has an independent kinematic chain, and angle sensors are employed to measure the finger movement including the flexion/extension and the adduction/abduction. The cable tension is controlled by a servo motor to transmit the interaction force to the fingers in real time.

The remainder of this paper is organized as follows. Section 2 provides a detailed description of the finger exoskeleton, including the mechanism and force reproduction system, followed by the motion capture model and the master–slave control strategy in Section 3. The performance verification experiments are in Section 4, whose results are analyzed and discussed in Sections 5 and 6, respectively. Finally, Section 7 draws the conclusion.

2 Mechanism of the Finger Exoskeleton

The finger exoskeleton contains the distal interphalangeal (DIP), proximal interphalangeal (PIP), and metacarpophalangeal (MCP) joints. Each finger joint possesses an independent four-link closed-loop kinematic chain. Figure 1 explains the principle of the finger exoskeleton mechanism. Taking the PIP joint as an example, Figure 1b demonstrates the geometric parameters relationship of the closed-loop kinematic chain. The O_PIP denotes the rotation center of the PIP joint, and an angle sensor is embedded in the hinge D. Different from the DIP and PIP joints, the MCP joint includes two DOFs (flexion/extension and adduction/abduction). Thus, another angle sensor is embedded at the base of the MCP joint to detect the adduction/abduction angle. Figure 1c embodies the geometric relationship of the closed-loop kinematic chain of the MCP joint. The O_MCP denotes the rotating center, and the angle sensor is coaxial with the rotating center of the adduction/abduction motion. The kinematic closed chains of the DIP, PIP, and MCP joint are cascaded to form a finger exoskeleton, which can adapt to different users without restricting human hand motion. Similarly, the thumb has the same four-link closed-loop kinematic chain structure as the DIP, PIP, and MCP joints of the index finger, and the corresponding joints respectively are interphalangeal (IP), MCP, and carpometacarpal (CMC) joints.

Figure 1

Finger exoskeleton mechanism: a The closed chain structure of the finger exoskeleton, b Kinematic parameters of the PIP joint, c Kinematic parameters of the MCP joint

In this research, we use the servo motor as the driver to realize the force reproduction for the master hand exoskeleton. Figure 2 shows the force reproduction system, which contains the driving unit, transmission unit, and detection unit. The servo motor and winding tiller constitute the driving unit. The cable, constraint holes, and constraint slots constitute the transmission unit. The reset spring, touch force sensor, and various components constitute the detection unit. To ensure the independence of each finger joint, the kinematic closed chains are connected in series by a cable. Taking the contact force of the slave hand as the input reference, the servo motor drives the winding tiller to change the cable length to transmit the force to the finger by restraining the motion of the exoskeleton. The detection unit converts the change of cable length into cable tension, which is detected by the FSS1500NST touch force sensor (Honeywell Inc, North Carolina, US). The reset spring provided the pre-tightening force for the cable, to improve the real-time responsiveness of the detection unit.

Figure 2

Force reproduction system of hand exoskeleton: a Assembly of the hand exoskeleton, b Transmission unit, c Detection unit

Based on the above-mentioned finger exoskeleton configuration and force reproduction system, an adaptive hand exoskeleton is proposed. The physical prototype is shown in Figure 3. The exoskeleton is attached to the elastic fiber gloves with an arc-shaped restraint ring. The total weight is below 300 g. The hand exoskeleton has a total 20 DOFs to achieve a range of motion covering the entire hand [18, 19].

Figure 3

Adaptive hand exoskeleton: a The prototype, b Worn on the hand, c Worn on the finger

3 Mathematic Model and Control

3.1 Joint Flexion/Extension Angle

According to the geometric relationship of the closed-loop kinematic chain mentioned in Section 2, the sensor for adduction/abduction motion is coaxial with the joint rotation center, and the angle of this motion can be directly sampled by the sensor. Still, the flexion/extension angle needs to be converted and solved by the geometric model.

As shown in Figure 1b, c, the angle θ_HPIP of PIP joint can be expressed as:

$$\theta_{{{\text{HPIP}}}} = \alpha_{{1{\text{PIP}}}} + \alpha_{{2{\text{PIP}}}} + \beta_{{1{\text{PIP}}}} + \beta_{{{\text{2PIP}}}} - \uppi ,$$

(1)

$$\left\{\begin{array}{c}\begin{aligned} \alpha_{{1{\text{PIP}}}} & = \arctan \left( {\frac{{H_{{1{\text{PIP}}}} }}{{W_{{1{\text{PIP}}}} }}} \right), \\ \alpha_{{2{\text{PIP}}}} & = \arctan \left( {\frac{{H_{{2{\text{PIP}}}} }}{{W_{{2{\text{PIP}}}} }}} \right), \\ \beta_{{1{\text{PIP}}}}& = \arcsin \left( {\frac{{L_{{4{\text{PIP}}}} \sin \theta_{{{\text{CPIP}}}} }}{{L_{{0{\text{PIP}}}} }}} \right), \\ \beta_{{2{\text{PIP}}}} & = \arcsin \left( {\frac{{L_{{3{\text{PIP}}}} \sin \theta_{{{\text{SPIP}}}} }}{{L_{{0{\text{PIP}}}} }}} \right), \\ \end{aligned}\end{array}\right.$$

(2)

where θ_HPIP is the joint angle; θ_SPIP is the desired measurement angle from the angle sensor; H_1PIP, H_2PIP, W_1PIP, W_2PIP, L_3PIP, L_4PIP are the lengths of the rods to be measured; θ_CPIP, L_0PIP can be obtained by the cosine theorem.

Furthermore, the PIP joint angle θ_HPIP can be expressed as a function of angle θ_SPIP.

$$\begin{aligned} \theta_{{{\text{HPIP}}}} & = \arctan \left( {\frac{{H_{{1{\text{PIP}}}} }}{{W_{{1{\text{PIP}}}} }}} \right) + \arctan \left( {\frac{{H_{{2{\text{PIP}}}} }}{{W_{{2{\text{PIP}}}} }}} \right) - \uppi \\ & \quad + \arcsin \left( {\frac{{L_{{4{\text{PIP}}}} \sin \theta_{{{\text{CPIP}}}} }}{{L_{{0{\text{PIP}}}} }}} \right) \\ & \quad + \arcsin \left( {\frac{{L_{{3{\text{PIP}}}} \sin \left( {\theta_{{{\text{SPIP}}}} - \theta_{{{\text{dPIP}}}} } \right)}}{{L_{{0{\text{PIP}}}} }}} \right). \\ \end{aligned}$$

(3)

Similarly, for the MCP joint, the joint angle θ_HMCP of its flexion/extension motion can be expressed as:

$$\begin{aligned} \theta_{{{\text{HMCP}}}} & = \alpha_{{1{\text{MCP}}}} + \beta_{{1{\text{MCP}}}} + \beta_{{2{\text{MCP}}}} - \frac{\uppi }{2} \\ & = \arctan \left( {\frac{{H_{{1{\text{MCP}}}} }}{{W_{{1{\text{MCP}}}} }}} \right) - \frac{\uppi }{2} \\ & \quad + \arcsin \left( {\frac{{L_{{4{\text{MCP}}}} \sin \theta_{{{\text{CMCP}}}} }}{{L_{{0{\text{MCP}}}} }}} \right) \\ & \quad + \arcsin \left( {\frac{{L_{{3{\text{MCP}}}} \sin \left( {\theta_{{{\text{SMCP}}}} - \theta_{{{\text{dMCP}}}} } \right)}}{{L_{{0{\text{MCP}}}} }}} \right). \\ \end{aligned}$$

(4)

The θ_dPIP and θ_dMCP in Eqs. (3) and (4) are the errors between the actual measurement angle and the desired measurement angle of the PIP and MCP joint angle sensors, respectively, which are caused by the measurement of the related parameters and mechanism design. To have the precise measurement, we calibrated the sensors before the tests.

3.2 Master–Slave Mapping

To ensure that the slave hand can follow the movement of the master hand in the teleoperation, it is necessary to establish an appropriate mapping between the corresponding joints of the master hand and the slave hand. The mapping between the angle change value Δθ_H of the master hand joint and the angle change value Δθ_R of the slave hand joint is:

$$\Delta \theta_{{\text{R}}} = k\Delta \theta_{{\text{H}}} ,$$

(5)

where k is the mapping coefficient.

The larger the k is, the more sensitive the slave hand will be. On the contrary, the smaller the k is, the more accurate the slave hand will be, which is more conducive to dexterous manipulations. Taking the single-finger exoskeleton as an example, the DIP, PIP, and MCP joints are bent to 30° and 60° with a step length of 1°. According to the geometric model in Section 3.1, the actual joint angle is calculated. The experiments are repeated several times, and the data were taken as the target value of the slave hand. In this study, the least square method is adopted to determine the parameter k.

The adaptive master hand exoskeleton has 20 DOFs. If a humanoid five-fingered dexterous hand with 20 DOFs is employed as the slave hand, the mapping relationship is as follows:

$$\left( \begin{array}{*{20}c} \Delta \theta_{{{\text{R}}1}} \\ \vdots \\ \Delta \theta_{{{\text{R}}20}} \\ \end{array} \right) = \left( {\begin{array}{*{20}c} {k_{1} } & {} & 0 \\ {} & \ddots & {} \\ 0 & {} & {k_{20} } \\ \end{array} } \right)_{20 \times 20} \left( \begin{array}{*{20}c} \Delta \theta_{{{\text{H}}1}} \\ \vdots \\ \Delta \theta_{{{\text{H}}20}} \\ \end{array} \right).$$

(6)

3.3 Master–Slave Control Strategy

To implement the slave hand to complete the complex operation like the human hand [20, 21]. Based on the force-position double-loop control method, a master–slave control strategy is designed, which is shown in Figure 4.

Figure 4

Diagram of the master–slave control strategy

The master side is the proposed adaptive hand exoskeleton, and the slave side is the humanoid five-fingered dexterous hand with the same number of DOFs. The position control loop realizes the following movement of the slave hand, and the force control loop realizes the force reproduction of the master hand. When the slave hand moves in free space and has no force feedback, the reference input of the master hand is the preset initial tension value provided by the reset spring. When the slave hand interacts with the object, the contact force is the reference input for the master hand. In Figure 4, Δθ_H is the angle variation of the master hand joint. Δθ_R is the angle variation of the slave hand joint. F_e is the pressure value of the fingertip of the slave hand. k_s is the tension conversion coefficient. F_p is the initial tension value of the cable. F_m denotes the tension value for the master hand.

4 Experiments

4.1 Interaction Motion Tests

Before the teleoperation experiments, the interactive performance between the human and the adaptive exoskeleton is tested. Figure 5 shows the motion capability of flexion/extension and adduction/abduction after wearing the hand exoskeleton (Additional file 1: Video S1).

Figure 5

Interaction motion tests of the hand exoskeleton (the blue dot represents the joint): a Four fingers flexion/extension, b Four fingers adduction/abduction, c Thumb flexion/extension, d Thumb adduction/abduction

It can be observed from Figure 5a, b that during the movement of the four fingers, the exoskeleton is always kept attached to the finger, with neither movement interference between the adjacent finger exoskeletons, nor mechanical constraints. The movement process of the thumb is more complicated. As shown in Figure 5c, d, when the thumb CMC joint performs flexion and extension, significant adduction and abduction happened. In the CMC joint, adduction/abduction is accompanied by flexion/extension, and the two motions are coupled. However, the exoskeleton can still be admirably adapted to the thumb joints (Additional file 2: Video S2).

4.2 Grasping Tests

The grasping tests are further conducted based on the Feix grasping classification method [22, 23]. As shown in Figure 6, the grasping ability of the exoskeleton after wearing is verified through various actions such as powerful grasping, intermediate grasping, and precise gripping. The results show that the master hand exoskeleton has satisfactory adaptability.

Figure 6

Grasping verification based on Feix classification: a Thumb abduction with strong grab force, b Thumb abduction with middle grab force, c Thumb abduction with small grab force, d Thumb adduction with strong grab force, e Thumb adduction with middle grab force, f Thumb adduction with small grab force

4.3 Teleoperation Tests

The teleoperating experimental platform is shown in Figure 7a. The fully driven uncoupled humanoid five-fingered dexterous hand (FDBM-Hand) is employed as the slave hand, as shown in Figure 7b [24]. The FDBM-Hand is composed of five modular fingers and a palm, which has the same number of DOFs as the master hand exoskeleton. Each finger joint is independently controlled by PAM.

Figure 7

The experimental platform of the teleoperating system: a The control diagram of the teleoperating system, b The fully driven humanoid five-fingered dexterous slave hand

On the teleoperation experimental system mentioned above, the following experiments are conducted.

• (1) Motion capture experiment

  Taking the index finger of the master hand exoskeleton as an example, the DIP, PIP, and MCP joints bent to 30° and 60° in turn. Multiple experiments are conducted to demonstrate repeatability.

• (2) Tension control experiment

  To improve the capability of the response of the master hand exoskeleton to the contact force instantaneously, an initial tension of the cable is preset. Verifying the control effect of the tension of the cable is a necessary process to determine that the adaptive hand exoskeleton can accurately reproduce the slave hand contact force information. In this experiment, the flexion/extension of the index finger is taken as an example, and the tension of the cable is controlled during the dynamic process.

• (3) Master–slave experiment in free space

  In this experiment, the subject wears the exoskeleton and performs various gestures in free space to observe the motion tracking performance of the slave hand.

• (4) Master–slave experiment in constrained space

  In constrained space, the force reproduction performance of the master hand is verified. The master hand teleoperated the index finger to interact with a sponge ball. The contact force can be obtained from the pressure sensor and feedbacked to the master hand in real-time.

5 Results

5.1 Accuracy of Joint Motion Capture

Tables 1 and 2 show the mean values and errors of the measured angles of the index finger DIP, PIP, and MCP of the master hand under the bending state of 30° and 60°. The errors of the joint angles are all within ± 3°.

Table 1 Joint angle test with input reference of 30°

Table 2 Joint angle test with input reference of 60°

5.2 Cable Tension Control

Figure 8 shows the curve of the joint angle and the tension of the cable. With the change of the index finger joint angle, the pressure data detected by the sensor exhibited a small range of fluctuations near the initial tension value. When the slave hand moved without any contact with the environment, the master hand still maintained a stable output tension at any time, which proves that the force reproduction system can achieve the desired effect.

Figure 8

The experimental curve of cable tension control

5.3 Master–Slave Follow in Free Space

In the master–slave follow experiment in free space, the master hand changed different gestures, and the snapshots of synchronous mapping to the slave hand are shown in Figure 9. It can be seen that the gestures reproduced by the slave hand are consistent with those of the master hand.

Figure 9

Master–slave mapping in free space: a Gesture with single finger, b Two fingers, c Three fingers, d Four fingers, e Five fingers

5.4 Master–Slave Teleoperation in Constraint Space

Figure 10 shows the master–slave teleoperation in the constraint space, which is divided into five steps.

• Step I: Approaching. The slave hand moved in free space and gradually approaches while not in contact with the sponge ball. At this stage, there is no contact force on the slave hand, and the master hand only outputs the initial tension.

• Step II: Extrusion. The slave hand starts to contact with the sponge ball and gradually squeeze the sponge ball. The interaction force of the slave hand gradually increases. At the same time, the tension of the cable follows the dynamic process of the slave hand.

• Step III: Holding. The slave hand continues to squeeze the sponge ball and hold it for some time. At this time, the contact force of the slave hand and the tension of the cable of the master hand only have a small fluctuation.

• Step IV: Releasing. The sponge ball is gradually released by the slave hand. At this time, the contact force of the slave hand and cable tension of the master hand both decrease rapidly.

• Step V: Disengagement. The slave hand is completely out of contact with the sponge ball and returned to free space. At this time, the output interaction force from the hand returns to zero, and the cable tension of the master hand is also restored to the initial tension.

Figure 10

Master–slave interaction control experiment: a The five steps of experiment, b The experimental curves

6 Discussion

The primary problem to be solved for the hand exoskeleton is that it should not produce mechanical constraints to the motion of the human hand. The proposed adaptive master hand exoskeleton has 20 DOFs and covers the entire hand motion. It is observed in the interactive motion and grasping tests that after wearing the adaptive hand exoskeleton, the subject completes different joint motions and grasping actions, and all of the actions are not constrained, which proves that the hand exoskeleton can be adapted to the human hand.

Then, the master hand maps the motion of human fingers to the slave hand, and the coupled motion between the knuckles makes it challenging for the master hand to accurately obtain the angles of the finger joints. This paper proposed a novel finger exoskeleton based on the closed chain cascade structure, which constituted an independent four-link closed-loop kinematic chain with each finger joint. The advantage of this mechanism is that each kinematic closed chain only captures the corresponding joint angle, and the accurate joint angle capture model can be established without considering the coupling motion constraints between knuckles. It can be seen from the master hand joint motion experiments that the angle capture accuracy of each joint of the master hand index finger is within ± 3° after the geometric model conversion. Such errors can be considered to be acceptable.

Finally, the master hand is required to reproduce the contact force between the slave hand and its object. This paper used the servo motor as the driver for each finger exoskeleton and changed the length of the cable to control its tension. The detection unit is integrated at each fingertip to detect the tension of the cable, which controls the output of the drive unit instantaneously. To ensure the response speed of the master hand, an initial tension was preset. In the verification of the tension control experiment, with the change of the angle of the finger joints, the output tension value of the master hand is kept steady. In the master–slave interaction experiment in constrained space, we find that in the extrusion stage, with the increase of the interaction force, the deviation between the angle of the master hand joint and the slave hand joint angle increases gradually. Such a solution is due to the slave hand moving restricted immediately after touching the sponge ball. The slave hand feedbacks the contacting force to the master hand, and the finger further squeezes the sponge ball after sensing the contacting event. At this moment, the joint angle of the master hand still increases, while the joint angle of the slave hand tends to be stable. In the holding stage, the cable of the master hand and the contacting force of the slave hand fluctuated slightly. This is because the finger of the master hand is required to resist the contraction of the cable to hold the joint posture steady. Thus, both the master and slave controllers are constantly adjusted to adapt to the motion of the fingers.

The adaptive hand exoskeleton proposed in this paper meets the requirements of teleoperation for the master hand. However, there are still some issues to be further studied and improved in future work. Firstly, the hand exoskeleton only consists of the front four DOFs of each finger and does not consider the CMC joint. A new mechanism with the CMC joint will be more compliant with the movement of the human hand. Secondly, the developed hand exoskeleton only reproduces the force sensing at the finger end, and the human hand cannot accurately perceive the force from the knuckles of the hand, which is not conducive to the operator making correct decisions. In the future, we will optimize the sensor deployment and fully reproduce the interactive information from hands.

7 Conclusions

• (1) According to the anatomy of human hand, a novel finger exoskeleton configuration composed of a cascade of four-link closed kinematic chains is presented. As a result, the independent capture of each finger joint is obtained.

• (2) In the interest of much more intuitive feelings from the slave hand contacting force, a force reproduction system with three subunits is designed.

• (3) Based on the finger exoskeleton mechanism, an adaptive hand exoskeleton prototype is developed, which has 20 DOFs total and covers the entire hand range of motion.

• (4) With sets of experiments, interaction motions and grasping tests are carried out and the exoskeleton shows satisfactory wearable adaptability. The motion capture experiments verified the accuracy of the force reproduction system.

• (5) The experiments of master–slave follow in free space and master–slave teleoperation in constrained space are implemented. The results proved the stability and reliability of the master–slave system.