- Original Article
- Open access
- Published:

# Spacecraft Pose Estimation Based on Different Camera Models

*Chinese Journal of Mechanical Engineering*
**volumeÂ 36**, ArticleÂ number:Â 63 (2023)

## Abstract

Spacecraft pose estimation is an important technology to maintain or change the spacecraft orientation in space. For spacecraft pose estimation, when two spacecraft are relatively distant, the depth information of the space point is less than that of the measuring distance, so the camera model can be seen as a weak perspective projection model. In this paper, a spacecraft pose estimation algorithm based on four symmetrical points of the spacecraft outline is proposed. The analytical solution of the spacecraft pose is obtained by solving the weak perspective projection model, which can satisfy the requirements of the measurement model when the measurement distance is long. The optimal solution is obtained from the weak perspective projection model to the perspective projection model, which can meet the measurement requirements when the measuring distance is small. The simulation results show that the proposed algorithm can obtain better results, even though the noise is large.

## 1 Introduction

With the development of space technology, an increasing number of space missions involve the relative position measurement of two spacecraft [1,2,3,4], such as space assembly, space satellite repair, fuel injection, satellite capture and tracking, and space interception. The measurement of the spacecraftâ€™s relative position is very important to maintain or change the spacecraft orientation in space to complete a space mission.

For relative position measurement, vision has some advantages in terms of weight, volume, power consumption, and equipment cost [5,6,7,8]. In the smart-OLEV mission [9, 10], the SMART-1 platform uses stereo cameras and lighting equipment to provide better measurement data within 5 m, but pointing data are only provided at a long distance. The Argon vision system is divided into long-distance and short-range vision sensors [11, 12], which select different field-of-view sensors for different distances. The natural image feature recognition system developed by the Johnson Space Center USES generates a 3D model of the target under test to calculate the relative pose [13, 14]. Its measurement accuracy is proportional to the relative distance. The measurement system requires to measure the relative pose information of two spacecraft at different distances for the control system or other systems, and the relative distance of the spacecraft varies significantly, reaching more than 20 times. Therefore, spacecraft pose estimation has the following characteristics.

(1) When the two spacecraft are relatively distant, the depth information of the feature points on the target spacecraft and the distance between the feature points are less than the relative distance between the two spacecraft.

(2) Because the focal length of the camera is fixed, the accuracy of the feature point extraction decreases with an increase in the relative distance between the two spacecraft.

Based on the reasons above, when the two spacecraft are far apart, the pose measurement accuracy will be reduced. At present, two main algorithms are used for pose estimation.

(1) Cooperative space measurement. The first is an analytical algorithm based on the perspective projection camera model, such as perspective-n-point [15,16,17] and direct linear transformation [18,19,20]. Using these algorithms, the pose of spacecraft can be solved directly. However, the accuracy of the spacecraft pose obtained using the analytical algorithm is unsatisfactory. and optimization algorithms based on nonlinear camera models, such as Gaussâ€“Newton, Levenbergâ€“Marqurdt, and orthogonal iteration algorithms [21,22,23]. These algorithms require a good initial solution for optimization. Therefore, we aim to obtain high-precision analytical solutions using analytical algorithms. (2) Based on noncooperative space measurement, the transformation of the pose is calculated by using pattern matching and 3D point cloud technology [24,25,26,27,28].

In this study, a spacecraft pose estimation algorithm based on the target geometric constraints of the spacecraft outline is proposed. To reduce the influence of distance on measurement accuracy, this study simplifies the camera measurement model. The simulation results show that the proposed algorithm has an image feature error of 0.1 pixel to 1 pixel from 1Â m to 20Â m.

## 2 Pose Estimation Algorithm

Spacecraft pose estimation is based on the relationship between the target spacecraft points and the corresponding image points. The relative pose of the target spacecraft coordinate system and camera coordinate system is calculated by using the multipoint correspondence relationship (Figure 1).

The mapping relations between the target spacecraft point and the image point can be described by two mathematical mappings: 1) rigid transformation and 2) camera model. In the former, the space points in the space coordinate system and camera coordinate system follow a rigid body transformation, namely, rotation and translation transformations. Because the camera is installed on the tracker spacecraft, the relative pose relationship between the tracker spacecraft and the target spacecraft can be described by the pose relationship between the target spacecraft and the camera coordinate system. In the latter, the relationship between 3D space points in the camera coordinates and the projection 2D image points on the camera image plane is considered.

### 2.1 Algorithm Model

To construct the spacecraft pose estimation algorithm model, four coplanar symmetry points are used to calculate the spacecraft pose. The target spacecraft coordinate system is *O*_{s}-*xyz*. There are four points for \({\varvec{P}}_{i}^{s} ,\) \(i = 1, \cdots ,4\). The four points in the target spacecraft coordinates are

where *a*, *b*, *c*, *d* are known values and the relationship between \({\varvec{P}}_{i}^{s}\) and the points in the camera coordinate system \({\varvec{P}}_{i}^{c}\) is given by

where

where *I* is the first row of the rotation matrix of \({\varvec{R}}_{3 \times 3}\), *J* is the second row, and *K* is the third row.

### 2.2 Camera Model

The fixed-focus-lens camera model can be simplified to a single-lens model. According to the optics principle, space points \(P_{i}^{c} (x_{i}^{c} ,\;y_{i}^{c} ,\;z_{i}^{c} )\), image points \(p_{i} (u_{i} ,\;v_{i} )\), and the camera origin \(O_{{\text{c}}}\) are located on the same line. Therefore, the camera model is called the pinhole camera model, which is also known as the perspective projection model.

where *f* is the camera focal distance. The spacecraft pose estimation model is

Form Eq. (2), we can obtain the relationship between \(z_{i}^{c}\) and \(t_{z}\):

Finally, we have

According to the symmetry properties of points, we have

where

### 2.3 Simplified Model

When the two spacecraft are relatively distant, the accuracy of the image feature extraction is low, and the depth information of the feature points to the target spacecraft can be ignored. The camera model can be approximated by a simplified perspective projection model [29,30,31]. Consequently, we obtain

Simplified perspective projection refers to the projection on a plane parallel to the imaging plane through the origin of the target spacecraft. Therefore, it ignores the depth of the target spacecraft point relative to the origin of the target spacecraft. When the two spacecraft are relatively distant, the neglect error is insignificant. From Eq. (8), we have

where *k*_{i} can be calculated by image points.Â Equation (7) contains nine variables and six equations. Thus, it cannot be solved directly. The rotation matrix ** R** has the following constraints:

From the first, third, and sixth equations of Eq. (12), we can obtain

From Eqs. (8) and (13), we obtain

Eq. (14) is a quartic equation. Therefore, the number of roots is four. Two negative roots are removed according to the relationship between roots and coefficients, and two positive roots meet the following conditions:

Condition 2 can only be satisfied when the rotation angle is greater than 60Â°; therefore, the result of applying Condition 1 is selected.

Rotation matrix ** R** can be described by four quaternion parameters \((q_{0} ,q_{1} ,q_{2} ,q_{3} )\):

Assumed that

Form Eq. (14), we obtain

As a result, we have

### 2.4 Optimization Algorithm

The accuracy of spacecraft pose estimation based on simplified perspective projection is poor. Therefore, an iterative optimal algorithm was constructed to improve the solution accuracy. The optimal algorithm for improving the accuracy of spacecraft pose estimation is shown in Figure 2.

In the algorithm, \(R^{j} = \left[ {\begin{array}{*{20}c} {I^{j} } & {J^{j} } & {K^{j} } \\ \end{array} } \right]^{{\text{T}}}\) and \(T^{j} = \left[ {\begin{array}{*{20}c} {t_{x}^{j} } & {t_{y}^{j} } & {t_{z}^{j} } \\ \end{array} } \right]^{{\text{T}}}\),

## 3 Experimental Section

The simulation experiment parameters were set as follows. The focal distance of the camera was 12 mm. The pixel size was 7.4 Î¼m Ã— 7.4 Î¼m. The rotation matrix and the translation vector were \([\varphi ,\;\theta ,\;\psi ] = [30^\circ ,\;30^\circ ,\;30^\circ ]\) and \(T = [0.5t_{z} ,\;0.5t_{z} ,\;t_{z} ]\)Â (m), respectively, where \(t_{z} = 1 - 20.\) The four points in the target spacecraft coordinates were

Simulation experiments verified the proposed algorithm in the following three aspects: 1) The optimization algorithm was analyzed without noise. 2) The relationship between the estimation accuracy and distance was analyzed with a mean value of 0 and a standard deviation of 0.1 pixel Gaussian noise. 3) The relationship between the estimation accuracy and distance was analyzed with a mean value of 0 and a standard deviation of 1 pixel Gaussian noise.

The simulation results are shown in Figures 3 and 4. The spacecraft pose estimation error is large, based on the simplified perspective projection, and the optimization algorithm based on the camera model effectively reduces the measurement error. After 10 iterations, the attitude errors are less than 0.42Â°, and the position errors are less than 4 mm.

Figures 5 and 6 show the estimation accuracy with a mean value of 0 and a standard deviation of 0.1 pixel noise. When \(t_{z}\) is 10 m, the attitude error is less than 0.36Â°, and the position error is less than 19.5 mm. When \(t_{z}\) is 20 m, the attitude error is less than 0.65Â°, and the position error is less than 117 mm. The maximum pose error occurs when \(t_{z} = 1\) m. This is mainly because the initial relative position accuracy is low based on the simplified perspective projection model.

Figures 7 and 8 show the estimation accuracy with a mean value of 0 and a standard deviation of 1 pixel noise. When \(t_{z}\) is 10 m, the attitude errors are less than 3Â°, and the position errors are less than 0.35 m. When \(t_{z}\) is 20 m, the attitude errors are less than 7.5Â°, and the position errors are less than 1 m.

## 4 Conclusions

To meet the requirements of pose estimation accuracy for spacecraft relative distance change from far to near, we propose a model based on two different camera models. In this model, the initial value of the spacecraft pose is calculated by a simplified perspective projection model, and the results are further optimized by the perspective projection model. The simulation results show that the errors of pose estimation are less than 0.8Â° and 117 mm when the image points have 0 mean and 0.1 pixel standard deviation. Further, the errors of pose estimation are less than 7.5Â° and 1 m when the image points have 0 mean and 1 pixel standard deviation. The estimation accuracy can satisfy the requirements of spacecraft missions.

## References

M Balch, D Tandy. A pose and position measurement system for the Hubble Space Telescope servicing mission.

*Proceedings of SPIE - The International Society for Optical Engineering*, 2007: 65550F.M Xu, Q Qu, Y Dong, et al. Capturing a spacecraft around a flyby asteroid using hamiltonian-structure-preserving control.

*Communications in Nonlinear Science and Numerical Simulation*, 2020: 105500.A M Zou, K D Kumar, A Ruiter. Fixed-time attitude tracking control for rigid spacecraft.

*Automatica*, 2020, 113: 108792.C Yin. Multi-loop attitude tracking control for capturing spacecraft.

*Aerospace Control*, 2018, 36(01): 42-49. (in Chinese)J Peng, W Xu, B Liang, et al. Virtual stereo-vision pose measurement of non-cooperative space targets for a dual-arm space robot.

*IEEE Transactions on Instrumentation & Measurement*, 2019: 1-13.S Sharma, J Ventura, S D'Amico. Robust model-based monocular pose initialization for noncooperative spacecraft rendezvous.

*Journal of Spacecraft and Rockets*, 2018, 55(6): 1414-1429.G Liu, C Xu, Y Zhu, et al. Monocular vision-based pose determination in close proximity for low impact docking.

*Sensors*, 2019, 19(15): 3261.L Zhang, F Zhu, Y Hao, et al. Rectangular-structure-based pose estimation method for non-cooperative rendezvous.

*Applied Optics*, 2018, 57(21): 6164-6173.C Kaiser, F Sjoeberg, J M Delcura, et al. SMART-OLEV-An orbital life extension vehicle for servicing commercial spacecrafts in GEO.

*Acta Astronautica*, 2008, 63(1-4): 400-410.F Sellmaier, T Boge, J Spurmann, et al. On-orbit servicing missions: Challenges and solutions for spacecraft operations.

*Spaceops Conference*, 2010, 2: 1816-1826.J M Galante, J V Eepoel, M Strube, et al. Pose measurement performance of the argon relative navigation sensor suite in simulated flight conditions.

*Occupational Ergonomics, Minneapolis, Minnesota*, 2012. https://doi.org/10.2514/6.2012-4927.B J Naasz, E J Van, S Z Queen, et al. Flight results from the HST SM4 relative navigation sensor system.

*33rd Annual AAS Rocky Mountain Guidance and Control Conference*, Breckenridge, CO, 2010, 137: 723â€“744.X Peng, Z Sun, M Chen, et al. Robust noncooperative attitude tracking control for rigid bodies on rotation matrices subject to input saturation constraint.

*International Journal of Robust and Nonlinear Control*, 2021, 32(3): 1583-1603.R Volpe, M Sabatini, G B Palmerini. Pose and shape reconstruction of a noncooperative spacecraft using camera and range measurements.

*International Journal of Aerospace Engineering*, 2017, 2017(2): 1-13.V Lepetit, F Moreno-Noguer, P Fua. EPnP: An accurate O(n) solution to the PnP problem.

*International Journal of Computer Vision*, 2009, 81(2): 155-166.P Chen, G Hu, J Cui. Extended gravitational pose estimation.

*Optik - International Journal for Light and Electron Optics*, 2014, 125(20): 6106-6112.Q Yu, G Xu, W Dong, et al. Solving the perspective-three-point problem with linear combinations: An accurate and efficient method.

*Optik - International Journal for Light and Electron Optics*, 2020, 228(3): 165740.L Quan, Z Lan. Linear N-point camera pose determination.

*IEEE Trans. Pattern Anal. Mach. Intell.*, 1999, 21(8): 774-780.R Galego, A Ortega, R Ferreira, et al. Uncertainty analysis of the DLT-Lines calibration algorithm for cameras with radial distortion.

*Computer Vision & Image Understanding*, 2015, 140: 115-126.A Nagano. Three-dimensional videography using omnidirectional cameras: An approach inspired by the direct linear transformation method.

*Journal of Biomechanics*, 2021, 128: 110722.W Zhang, G Xu, Y Cheng, et al. Research on orthogonal iteration algorithm of visual pose estimation for UAV landing.

*2018 IEEE CSAA Guidance, Navigation and Control Conference (GNCC)*, Xiamen, China, 2018: 1-6.G Hu, Z Zhou, J Cao, et al. Non-linear calibration optimisation based on the Levenbergâ€“Marquardt algorithm.

*IET Image Processing*, 2020, 14(7): 1402-1414.C Sun, H Dong, B Zhang, et al. An orthogonal iteration pose estimation algorithm based on an incident ray tracking model.

*Measurement Science & Technology*, 2018. https://doi.org/10.1088/1361-6501/aad014.J Li, Y Zhuang, Q Peng, et al. Pose estimation of non-cooperative space targets based on cross-source point cloud fusion.

*Remote Sensing*, 2021, 13(21): 4239.L Zhang, F Zhu, Y Hao, et al. Optimization-based non-cooperative spacecraft pose estimation using stereo cameras during proximity operations.

*Applied Optics*, 2017, 56(15): 4522.G Zhao, S Xu, Y Bo. LiDAR-based non-cooperative tumbling spacecraft pose tracking by fusing depth maps and point clouds.

*Sensors*, 2018, 18(10): 3432.Q Wei, Z Jiang, H Zhang. Robust spacecraft component detection in point clouds.

*Sensors*, 2018, 18(4): 933.L Liu, G Zhao, Y Bo. Point cloud based relative pose estimation of a satellite in close range.

*Sensors*, 2016, 16(6): 824F Dornaika, C Garcia. Pose estimation using point and line correspondences.

*Real-Time Imaging*, 1999, 5(3): 215-230.I Shimshoni, R Basri, E Rivlin . A geometric interpretation of weak-perspective motion.

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 1999, 21(3): 252-257J H Byun, T D Han. PPAP: Perspective projection augment platform with Panâ€“Tilt actuation for improved spatial perception.

*Sensors*, 2019, 19(12): 2652.

## Acknowledgements

The authors sincerely thanks to Professor Naiming Qi of Harbin Institute of Technology for his critical discussion and reading during manuscript preparation.

## Funding

Supported by National Natural Science Foundation of China (Grant No. 12272104).

## Author information

### Authors and Affiliations

### Contributions

LM was in charge of the whole algorithm and wrote the manuscript; NQ provided suggestions and rectification ideas; ZZ assisted with simulation analysis. All authors read and approved the final manuscript.

### Authorsâ€™ Information

Lidong Mo, is currently a PhD candidate at *School of Astronautics, Harbin Institute of Technology, China*.

Naiming Qi, is currently a professor at *School of Astronautics, Harbin Institute of Technology, China*. He received his PhD degree from *Harbin Institute of Technology, China*, in 2001.

Zhenqing Zhao, is currently a teacher at *Northeast Agricultural University, China*.

### Corresponding author

## Ethics declarations

### Competing Interests

The authors declare no competing financial interests.

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Mo, L., Qi, N. & Zhao, Z. Spacecraft Pose Estimation Based on Different Camera Models.
*Chin. J. Mech. Eng.* **36**, 63 (2023). https://doi.org/10.1186/s10033-023-00884-8

Received:

Revised:

Accepted:

Published:

DOI: https://doi.org/10.1186/s10033-023-00884-8