Flexible control and trajectory planning of medical two-arm surgical robot

With the continuous development of medical technology and the progress of robot technology, more and more robots are applied in the medical industry (Zhong et al., 2019). The flexible control and trajectory planning of medical dual-arm collaborative robots are crucial technologies enabling these robots to perform various complex tasks in the medical field, such as surgical assistance (Wu et al., 2019) and rehabilitation training (Culmer et al., 2009). Flexible control (Chalhoub and Ulsoy, 1987) refers to the ability of the robot control system to adapt to external environments and task requirements. For example, in the medical surgery, the responsive implementation, real-time control systems are crucial. These systems are adept at detecting and adjusting to minute fluctuations within the patient internal environment, thereby ensuring both the precision and safety of surgical procedures. Additionally, these adaptable control systems make robots more skilled and flexible when they do difficult jobs. This is most noticeable when robots have to do tasks when they are working closely with people. Flexible control is often closely related to trajectory planning in medical dual-arm robots (Kojima and Kibe, 2001; Kojima, 2002).

Trajectory planning refers to determining the robot motion trajectory in the workspace to enable it to accomplish specified tasks (Asada et al., 1990). In medical dual-arm robots, trajectory planning must consider factors such as anatomical structures of the surgical area, surgical objectives, and safety to ensure that the robot motion can reach the expected positions accurately and safely.

In the standard operational procedures for industrial robots, ensuring both the robot functionality and the staff safety are paramount. To achieve this, a physical barrier is commonly implemented. This barrier serves to segregate the robot from the surrounding area or to delineate the operational zones of both the staff and the robot. This strategic separation effectively prevents any direct physical contact between the staff and the robotic equipment. In the medical sector, however, there is a distinct preference for robots that can collaborate closely with human operators. This necessitates the capability of robots to execute a multitude of tasks securely within the intricate and demanding conditions of surgical settings. The robots must be designed to operate harmoniously with medical professionals, ensuring that they can contribute effectively to the high-precision requirements of healthcare without compromising safety. Collision protection of medical dual-arm collaborative robots is critical to ensuring robot safe operation when working with humans. This protection typically employs various technologies and methods:

1. Sensor technology (Zeng and Bone, 2013; Popov et al., 2017; Li et al., 2020): Sensors installed at various vital parts of the robot, such as arms and around the body, detect proximity or collisions. These sensors can be proximity, photoelectric, or pressure sensors, among others.

2. Vision systems (Mohammed et al., 2017): Vision systems using cameras or laser scanners are utilized to detect the surrounding environment and human positions and timely identify potential collision hazards.

3. Collision detection algorithms (Cao et al., 2019; Han et al., 2019): Software algorithms analyze data provided by sensors and vision systems to determine the presence of collision risks and take corresponding measures to avoid collisions.

Adding sensors to the above collision detection method will increase the cost of the robot system, while sensorless collision detection will use acceleration information, which will lead to the introduction of interference and error.

In the robot arm kinematics, the research approaches (Venanzi and Parenti-Castelli, 2005; Pandey and Zhang, 2012) for kinematics modeling and parameter identification have reached a high level of maturity. Rectifying the kinematics parameters based on the general error model can directly enhance the accuracy of the robot arm end position. However, the following drawbacks may emerge during the process: a significant amount of calculation, a long period for identification parameter iteration, which severely affects the motion velocity. During the medical robot operating process, it is challenging to employ sensors to detect and provide real-time feedback on the end pose. Nevertheless, the artificial neural network (ANN) boasts strong self-learning and adaptive capabilities (Aoyagi et al., 2012; Angelidis and Vosniakos, 2014). In recent years, ANN has been extensively utilized in domains such as system optimization and intelligent control, among which the BP (Back Propagation) neural network is a multi-layer feedforward network trained in accordance with the reverse propagation algorithm. Utilizing the algorithm based on the neural network to estimate the pose at the end of the manipulator can circumvent certain issues in the process of parameter calibration.

This study delves into neural network technology, analyzing the inverse kinematics problem (IKP) and forward kinematics problem (FKP). The current FKP solving strategy relies on iterative methods, which incur high computational costs and long processing times and cannot achieve optimal real time operations. This paper proposes a neural network utilizing an improved form of multilayer perceptron for backpropagation learning to enhance the accuracy of solving the mechanical arm forward kinematics problem to the desired level and achieve real-time solutions.

Furthermore, the study investigates the establishment of NURBS curve-related techniques. It integrates NURBS curves into trajectory interpolation of medical dual-arm collaborative robots capable of handling any geometric shape. Additionally, this paper introduces compensation and variable damping design into the collision detection method based on the second-order momentum torque observer to improve robot collision detection real-time accuracy.

In the rapidly evolving medical industry landscape today, robotic automation has gradually entered the realm of medical systems. Traditional assembly processes of medical assistive devices are increasingly unable to meet the demands of modern medical production. Simultaneously, there is a growing demand for robots with high safety, stability, and flexibility to assist in collaborative operations during medical device-assisted drug dispensing. Robots can replace many tasks in medical drug dispensing. Manual solutions such as medication dispensing, assisting in guiding needles, and tumor resection using medical assistive devices must be updated. Medical dual-arm collaborative robots can perform complex drug dispensing processes, as depicted in Figure 1. The application of the medical dual-arm collaborative robot software platform in the medical assistive industry offers unparalleled advantages. These robots facilitate the drug dispensing process and enable collaboration between workers and robots. Integrating manual drug dispensing tasks with automated production addresses the labor-intensive operations in the medical industry. Moreover, the arms of these robots must possess the functionality for coordinated dual-arm operations to ensure synchronous and precise coordination and collision prevention.

The classification of robot interaction control is illustrated in Figure 2. Common control methods (Niku, 2020) mainly include force/position hybrid control and impedance/admittance control. In several studies (Hogan, 1985; Anderson and Spong, 1988) on impedance control, a prescribed fixed passive impedance model is defined, and efforts are then focused on addressing challenges such as dealing with uncertainties. Research within this framework typically adopts either learning-based (Sharifi et al., 2021) or adaptive impedance control (Yu et al., 2019). However, assuming that a fixed impedance model is no longer sufficient to describe specific applications, such as explosive movements or Human-Robot Interaction (HRI). Therefore, adopting variable impedance control (Ikeura and Inooka, 1995) must be considered for robot interaction control. However, adjusting impedance parameters to provide optimal impedance characteristics is more effective in enhancing interaction performance, which is necessary for essential applications such as HRI. Physically informed HRI has been proposed in adaptive impedance kinematics learning. In this study (Dong and Ren, 2017), robots adapt their movements by learning tasks to predict the intentions of their partners. Optimal impedance adaptation in constrained motion HRI has been investigated (Sun et al., 2019). Continuous critic learning is employed for interaction control, followed by obtaining the desired impedance as the optimal implementation to meet the control objectives. However, adaptive control is not appropriate for the control system with a high real-time demand, and it is deficient in the capacity to handle nonlinear issues. Fuzzy control. The fuzzy processing of the control parameters might cause the reduction of precision and the deterioration of the dynamic quality of the force interaction control.

Li et al. (Li et al., 2016) explores the cooperative kinematic control of multiple manipulators through distributed recurrent neural networks and offers a feasible approach to extend the existing outcomes on individual manipulator control by using recurrent neural networks to the circumstance of the coordination of multiple manipulators. Jin et al. (Jin et al., 2022) analyses a collaborative control problem of redundant manipulators with time delays and proposes a time-delayed and distributed neural dynamics scheme. Yang et al. (Yang et al., 2023) proposes an Extended Kalman Filter-incorporated Residual Neural Network-based Calibration (ERC) model for kinematic calibration.

As shown in Figure 3, the shoulder and elbow joints are designed with dual-degree mechanisms, differing only in motor arrangement. The motor achieves arm abduction through bevel gear transmission, while the offset motor achieves arm flexion and extension through spur gear transmission. The structural principles of the elbow joint are the same as those of the shoulder joint.

Peripheral components such as end effectors and dexterous hands are installed at the end, and backlash compensation is applied to the spur gear transmission system through an eccentric flange. The arm integrates driver boards for the elbow and wrist motors, facilitating later debugging and maintenance. Touch sensor modules can also be installed inside the arm to perceive human contact and respond with corresponding actions such as speech or facial expressions, enhancing interaction with humans.

The humanoid two-arm robot has 14 degrees of freedom and can be analyzed for one hand and the other hand. The optional position of the base coordinate system is not unique. Establishing a coordinate system in the above way can facilitate the analysis of the geometric relationship between each joint value and the robot pose. Compared with the traditional D-H method, the homogeneous transformation matrix between the adjacent joints of the robot can be intuitively obtained during the forward solution process. At the same time, the calculation amount of inverse kinematics analysis can be reduced to a certain extent (Liu et al., 2021). This method is suitable for fast modeling in engineering, can simplify the derivation of kinematic relations, facilitate the acquisition of kinematic equations, and make the results more intuitive. Since the arm position is decoupled from the attitude, the first four joints and the wrist are calculated separately to facilitate the inverse solution. The robot is divided into four freedom degrees ( θ1,θ2,θ3,θ4 ) at the lower part of the wrist and three degrees freedom ( θ5,θ6,θ7 ) at the end of the wrist joint. The forward kinematics is analyzed from the four joints at the base and three joints at the end. For the left hand, the transformation matrix from 1 to 7 can be obtained according to the spatial geometric relationship of each joint, as shown in Equation 1.

pi,ti ; By definition, the corresponding node vector is U=u0,u1,⋯,un+2k ; The specific expression is shown in Equation 9. {u0=u1=⋯uk=0un+k=un+k+1=⋯un+2k=1 (9)

Formula ui calculated by cumulative chord length method:

ui+3=ui+2+|pi−pi−1|∑|pi−pi−1|i=1,2,⋯,n−1 (10)

(2) The calculation process of matrix R elements is shown in Equations 8 and 11.

[1 a2b2c2 ⋱⋱⋱ anbncn 1]·[d1d2⋮⋮dn+1]=[e1e2⋮⋮en+1] (11)

(3) Control the calculation process of vertex di

The control vertex dl1 is calculated according to the following Equation 12.

[d1d2⋮⋮dn+1]=inv([1 a2b2c2 ⋱⋱⋱ anbncn 1])·[e1e2⋮⋮en+1] (12)

From the NURBS first and last points (control points), the Equation 13 can be obtained.

dl1dl2⋮dln+2dln+3=p0d1⋮dn+1pn (13)

(4) B spline basis function Niu has been calculated; The four basis functions of each segment B spline should be calculated according to the following formula, and the four basis functions Nj,3u corresponding to j=i−3,i−2,i−1,i should be calculated periodically for each segment, as shown in Equation 14.

Nj,3u=(ui+1−u)3ui+1−ui−2ui+1−ui−1ui+1−ui,j=i−3(ui+1−u)2u−ui−2ui+1−ui−2ui+1−ui−1ui+1−ui+u−ui−1ui+1−uui+2−uui+1−ui−1ui+1−uiui+2−ui−1+(ui+2−u)2u−uiui+2−ui−1ui+2−uiui+1−ui,j=i−2(ui−1−u)2ui+1−uui+2−ui−1ui+1−ui−1ui+1−ui+u−ui−1ui+2−uu−uiui+2−ui−1ui+2−uiui+1−ui+(ui−u)2ui+3−uui+3−uiui+2−uiui+1−ui,j=i−1(u−ui)3ui+3−uiui+2−uiui+1−ui,j=i (14)

(5) Subsection curve expression calculation process

Calculate the expression of each B spline curve according to the following Equation 15, which needs to be calculated periodically for each segment:

{Px(u)=∑j=i−3iωjpxjNj,3(u)∑j=i−3iωjNj,3(u)Py(u)=∑j=i−3iωjpyjNj,3(u)∑j=i−3iωjNj,3(u)Pz(u)=∑j=i−3iωjpzjNj,3(u)∑j=i−3iωjNj,3(u) (15)

At this stage, the B spline curve is fitted. Next, the curve data points need to be densified to enable the robot to walk to each densification point periodically. This densification is generally achieved through Taylor expansion.

3.3 Collision detection basics

Considering the influence of the external collision torque and the friction torque, the robot dynamic equation can be written as Equation 16. q,q˙,q¨ represent the robot joint angle, angular velocity and angular acceleration, respectively. Robot inertia matrix is D. Coriolis matrix is C. Gravitational term is G. The external impact is equivalent to the external torque of each joint is τe . Robot joint friction torque is τf . Robot motor drive torque is τ .

Dqq••+Cq,q•q•+Gq+τe+τf=τ (16)

As can be seen from the Equation 16, solving τerequires the acceleration, but it will bring noise. So we design an observer based on generalized momentum. The robot generalized momentum is defined as Equation 17.

We define an observation vector r. The torque gain matrix is K1 and K2. P̂ is the estimate of P. The second-order observer is shown in Equation 18.

r=K1∫0t−K2r+P̂−Pdt (18)

We optimize this second-order observer by introducing links such as Equation 19.

uf=∫0tτ−τf+CTq,q•q•−Gqdt−P (19)

Finally, the resulting observer is shown in Equation 20.

r=K1∫0t−K2r+P̂−Pdt+K∗uf (20) 4 Simulation analysis and experimental verification 4.1 Network verification

In the exploration of applying neural networks to solve the FKP, we initially collected the requisite datasets in accordance with established approaches. Subsequently, we employed BP neural network technology to train the model systematically, with the parameters of the joint motor serving as the input of the network and the coordinate position of the robot arm end as the output target.

Once the model training was accomplished, we randomly chose some data samples that were not encompassed in the training to test the prediction capability of the neural network. Based on the outcomes presented in Figure 8, we discovered that the neural network could precisely predict the coordinate position of the robot arm end. This finding thoroughly substantiates that the approach of leveraging the BP neural network to predict the position of the robot arm end is not only feasible but also highly accurate.

Figure 8. Comparison of end coordinate prediction results.

Additionally, this data-driven prediction method possesses stronger adaptability and flexibility compared to the traditional analytical method, and can handle the complex scenarios in practical applications more effectively.

4.2 Kinematics verification

Based on the characteristic parameters of the robotic arm links and the end effector spherical wrist, the correctness of both forward and inverse kinematics algorithms can be verified through curve acquisition in the driver software. The planner generates the trajectory curve of the robot end effector, and joint values are obtained through inverse kinematics. Then, spatial positions are calculated through forward kinematics, plotting numerous position points into a spatial curve. The degree of overlap between the two curves is compared.

Sinusoidal trajectories and spatial circular arc trajectories are separately applied to track trajectories using kinematic algorithms. As illustrated in Figures 9, 10, these are curves collected by the driver software.

Figure 9. Sine tracking curve.

Figure 10. Circular arc trace curve.

4.3 Trajectory planning verification

The NURBS curve functions as a generalized parameter interpolator, maintaining a uniform feed rate for most of the interpolation process and ensuring that each interpolation point falls within a specified error range. As shown in Figure 11, the interpolator avoids sharp corners and feed-sensitive angles in the curve, thereby mitigating high-frequency components or frequencies matching the machine inherent frequency in the interpolation trajectory, reducing high jitter. In the trajectory planning method, a forward-looking module detects sharp corners of the NURBS curve. An acceleration-deceleration method then adjusts the feed rate at these corners to meet error requirements and accommodate the robot acceleration and deceleration capabilities.

Figure 11. NURBS spline diagram.

4.4 Collision detection algorithm

To quickly verify the feasibility of the collision detection algorithm, we conducted simulation experiments using a two-degree freedom robot model for comparative research. As shown in Figure 12, the dashed line represents the simulation results of the unimproved observer (Equation 18). In contrast, the solid red and green lines represent the simulation results of the improved torque observer (Equation 20). The solid black line simulates the occurrence of a collision. It can be observed that after the collision occurs, the improved observer exhibits reduced overshoot and quick response speed.

Figure 12. Collision detection simulation.

4.5 True machine verification

This paper conducts experiments on flexible control and trajectory planning using the medical dual-arm collaborative robot produced by Siasun Corporation. Figure 13A depicts the experimental results of drug dispensing by the medical assistance dual-arm robot.

(1) Opening the bottle cap: (demonstrated using preopened medication, placing the opened aluminum cap into a box)

a. The robot left arm grabs the bottle while the right arm grasps the bottle opener.

b. The left arm places the bottle in the specified position and visually checks the notch position. If the position is incorrect, the left arm rotates the bottle to a specific position. Subsequently, the right arm grabs the bottle opener and quickly presses down to remove the aluminum cap from the medication using pressure.

c. The right arm returns the bottle opener to its original position, while the left arm lifts the bottle and places the pressed aluminum cap into the collection box. A visual system is introduced here to confirm whether the bottle cap has been successfully opened. An error message is prompted if it is still attached to the bottle or the opener.

d. After disposing of the aluminum cap, the bottle is placed back in the specified position.

(2) Medication dispensing demonstration:

a. The right arm grabs the syringe, as shown in Figure 13B, while the left grabs the plunger.

b. The two arms cooperate to demonstrate liquid suction from the vial and inject the liquid into the bottle, as shown in Figure 13C.

Figure 13. (A) Medical robot medicine dispensing. (B) Medical robot medicine dispensing. (C) Medical robot medicine dispensing.

5 Conclusion

This paper addresses the challenges of flexible control and trajectory planning in medical dual-arm collaborative robots. It proposes a neural network based kinematic solver as a universal approximator to resolve the kinematic problems of humanoid dual-arm robots while ensuring solution accuracy. Compared to other numerical and geometric methods for solving equation systems, the neural network based kinematic solver directly obtains reasonable solutions in the workspace when the number of unknowns exceeds the number of equations, providing a concise selection approach. Regarding trajectory planning, we propose a comprehensive interpolation scheme based on NURBS curve interpolation. The NURBS spline fitting method is employed to further smooth the interpolated feed curves, with repeated checks for chord error during interpolation to restrict it within a specified error range. Additionally, an improved second-order momentum torque observer is designed to accurately detect external collisions without external sensors. This observer operates without requiring arm acceleration as input, effectively avoiding interference and errors caused by acceleration. Optimized observer design significantly reduces system overshoot, thereby enhancing collision detection accuracy.

Data availability statement

The data supporting this study’s findings are available from the corresponding author upon reasonable request.

Author contributions

YX: Writing – original draft. XZ: Writing – original draft. YJ: Writing – original draft, Writing – review & editing. YW: Writing – review & editing. HY: Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was funded by Basic Research on Stress induced Bone Growth of Porous Titanium and Its Potential Application in Spinal Fusion Cages (Project No.: 2023JH2/101700136), Basic and Applied Research Project of Liaoning Provincial Science and Technology Planning.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Angelidis, A., and Vosniakos, G. C. (2014). Prediction and compensation of relative position error along industrial robot end-effector paths. Int. J. Precis. Eng. Manuf. 15, 63–73. doi: 10.1007/s12541-013-0306-5

Crossref Full Text | Google Scholar

Aoyagi, S., Suzuki, M., Takahashi, T., Fujioka, J., and Kamiya, Y. (2012). Calibration of kinematic parameters of robot arm using laser tracking system: compensation for non-geometric errors by neural networks and selection of optimal measuring points by genetic algorithm. Int. J. Autom. Technol. 6, 29–37. doi: 10.20965/ijat.2012.p0029

Crossref Full Text | Google Scholar

Asada, H., Ma, Z. D., and Tokumaru, H. (1990). Inverse dynamics of flexible robot arms: Modeling and computation for trajectory control. J. Dyn. Sys., Meas., Control Jun 1990, 112: 177–185. doi: 10.1115/1.2896124

Crossref Full Text | Google Scholar

Cao, P., Gan, Y., and Dai, X. (2019). Model-based sensorless robot collision detection under model uncertainties with a fast dynamics identification. Int. J. Adv. Robot. Syst. 16:1729881419853713. doi: 10.1177/1729881419853713

Crossref Full Text | Google Scholar

Chalhoub, N. G., and Ulsoy, A. G. (1987). Control of a flexible robot arm: Experimental and theoretical results. J. Dyn. Sys., Meas., Control 109:4. doi: 10.1115/1.3143859

Crossref Full Text | Google Scholar

Culmer, P. R., Jackson, A. E., Makower, S., Richardson, R., Cozens, J. A., Levesley, M. C., et al. (2009). A control strategy for upper limb robotic rehabilitation with a dual robot system. IEEE-ASME T. Mech. 15, 575–585. doi: 10.1109/TMECH.2009.2030796