310;

at least one inertial sensor 330, disposed on one or both of the executing device 310 and the driving device 320 and configured to detect motion of the driving device 320 to generate inertial sensing data; and

a control device 340, connected to the at least one inertial sensor 330 and configured to perform at least a part of or all operations of the method for controlling the robotic apparatus according to the embodiments of the present disclosure.

In some embodiments, the executing instrument 311 may be a surgical instrument. In some embodiments, a surgical instrument may be an instrument that performs surgical operations (such as grasping, shearing, cutting, clamping, sewing, etc.) on a target object, such as surgical forceps, surgical scissors, high-frequency electric knives, suture needles or the like. In some embodiments, the surgical instrument may also be an instrument that assist in performing surgical operations, such as various types of image acquisition devices, endoscopes or the like. It should be understood that the surgical instruments are not limited to the above-listed examples, but may be any instruments related to performing surgery, and will not be exhaustively described herein.

In some embodiments, the executing device 310 may include a motor pack (MPK) for a surgical instrument and configured to drive the surgical instrument fixed on the executing device to perform various surgical operations

In some embodiments, the driving device 320 may include an insertion arm for the surgical instrument.

In some embodiments, the inertial sensor 330 includes two or more of a gyroscope, an accelerometer, and a magnetometer. In some embodiments, two types of the inertial sensor 330 may be employed, for example: a gyroscope and an accelerometer, a gyroscope and a magnetometer. Alternatively, more than two types of the inertial sensor 330 may be employed, for example: a gyroscope, an accelerometer, and a magnetometer.

In some embodiments, the control device 340 may be implemented by software, hardware, firmware, or a combination thereof. In some embodiments, the control device 340 may use circuits, which may include one or more of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a PLC, a microcontroller, and a microprocessor.

In some embodiments, the control device 340 may include at least one processor and a memory being in communication connection with the at least one processor. The memory is configured to store instructions, which, when executed by the at least one processor, causes the at least one process to perform at least a part of or all operations of the method for controlling the robotic apparatus according to the embodiments of the present disclosure.

The memory and the at least one processor are connected via a bus. The bus may include any number of inter-connected buses and bridges. The bus connects various circuits of the at least one processors and the memory together. The bus may also connect other various circuits such as peripheral devices, voltage regulators, power management circuits and the like together, which is well known in the art. The at least one processor is in charge of managing the buses and routine procedures and further providing various functions such as timing, peripheral interfaces, voltage regulation, power source management and other control functions. The memory is configured to store data used by the at least one processor to perform operations. The memory may include U disks, removable hard disks, Read Only Memory (ROM), Random Access Memory (RAM), magnetic diskette, compact disks or other storage mediums capable of storing program codes.

In some embodiments, the robotic apparatus further includes:

- a transmission device 350, connected to the driving device 320 and configured to move the driving device 320;
- a force providing device 360, connected to the transmission device 350 and configured to provide a driving force to the transmission device 350; and
- an adjusting device 370, connected to the force providing device 360 and configured to adjust one or both of a position and an orientation of the transmission device 350, and one or both of a position and an orientation of the force providing device 360.

In some embodiments, the transmission device 350 may include a parallel mechanism.

In some embodiments, the force providing device 360 may include one or more motors, for example, a compact motor drive. In some embodiments, the control device 340 may control the output angles of the force providing device 360.

In some embodiments, the adjusting device 370 may include a set-up joint (SUJ) disposed at a rear end of the robotic apparatus. As shown in FIG. 3, the robotic apparatus may be a surgical robot, and the transmission device 350 together with the force providing device 360 may serve as a robotic arm of the surgical robot. The set-up joint is disposed at the rear end of the robotic arm and connected to the robotic arm. The set-up joint can conduct one or more of operations of moving the robotic arm horizontally, moving the robotic arm vertically, and adjusting the orientation of the robotic arm, in order to adjust the robotic arm to be in the suitable orientation and position, thereby ensuring that the area of the target object on which surgical operations are performed is within the range of motion of the executing device 310, so that the distance between the robotic arm and the target object is suitable for surgical operations on the target object. Referring to FIG. 4, FIG. 4 shows a schematic flowchart of the method for controlling the robotic apparatus according to some embodiments of the present disclosure. As shown in FIG. 4, the method may be performed by the robotic apparatus, and the method 400 for controlling the robotic apparatus includes:

At S410, inertial sensing data is acquired from at least one inertial sensor disposed on one or both of the executing device and the driving device;

At S420, data fusion is performed on at least the inertial sensing data to obtain fused data of the robotic apparatus; and

At S430, an operation status of the robotic apparatus is determined based on the fused data, and the robotic apparatus is controlled in response to the operation status.

By using at least one inertial sensor disposed on one or both of the executing device and the driving device to acquire inertial sensing data and performing data fusion and analysis, precision of the detection performed on the robotic apparatus can be improved, the operation status of the robotic apparatus can be accurately analyzed, and corresponding control can be conducted. In this way, the reliability of the robotic apparatus can be improved. Compared with the traditional robotic apparatus, a plurality of sets of encoders can be eliminated, thereby greatly simplifying the structure of the robotic apparatus, reducing weight and volume of the robotic apparatus, and reducing costs.

In some embodiments, the inertial sensing data includes one or more of acceleration data, angular velocity data, and earth magnetic field direction data. In some embodiments, the acceleration data may be collected through detection by an accelerometer. In some embodiments, the angular velocity data may be collected through detection by a gyroscope. In some embodiments, the earth magnetic field direction data is collected through detection by a magnetometer. In some embodiments, the accelerometer, the gyroscope, or the magnetometer each sends collected respective inertial sensing data to the control device, and the control device performs data fusion calculations to obtain the fused data.

In some other embodiments, a complementary algorithm, an algorithm based on noise models, or an algorithm based on particle swarm optimization is used to perform the data fusion on at least the inertial sensing data.

The algorithms for the data fusion are used to fuse data based on optimized utilization of the inertial sensing data and control signals, including but not limited to inertial sensing data (such as acceleration data, angular velocity data, or direction data), output of the driving devices (such as torque output of a driver), input current of the force providing device (such as motor detection current), etc., to generate optimized deformation amount signals of the transmission device. In comparison, the complementary algorithm has the lowest computation complexity and average fusion performance. The algorithm based on noise models has slightly higher computation complexity and excellent fusion performance. The algorithm based on particle swarm optimization has the highest computation complexity and the best data fusion performance.

In some other embodiments, the operation S420 of performing the data fusion on at least the inertial sensing data, includes:

- calculating one or more of an actual pitching angle, an actual rolling angle and an actual yawing angle of the driving device based on at least the inertial sensing data.

The operation S430 of determining the operation status of the robotic apparatus based on the fused data, and controlling the robotic apparatus in response to the operation status, includes:

- determining whether one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle are synchronized with one or more respective target angles;
- in response to one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle being not synchronized with the one or more respective target angles, determining that the robotic apparatus is working abnormally; and
- in response to the robotic apparatus being working abnormally, controlling the robotic apparatus to stop working.

By determining whether one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle of the driving device are synchronized with a respective target angle, it can be determined whether the robotic apparatus is working abnormally. For example, referring to FIG. 3, in a surgical robot, the executing instrument 311 may be a surgical instrument, the executing device 310 may include a motor pack for a surgical instrument, the driving device 320 may include an insertion arm for surgical instrument, and the motor pack for the surgical instrument may be disposed on the track of the insertion arm for surgical instrument. The insertion arm for surgical instrument drives the motor pack for the surgical instrument on which the surgical instrument is mounted to move on the track to perform corresponding surgical operations. It can be seen that the executing device 310 and the driving device 320 are consistent in pitching angle, rolling angle, or yawing angle. Thus, either the executing device 310 or the driving device 320 may be provided with at least one inertial sensor, and the fused data may be obtained by fusing the data collected through detection by the at least one inertial sensor. Alternatively, the executing device 310 and the driving device 320 are each provided with at least one inertial sensor for concurrent detection. In this way, redundancy of the inertial sensors can be achieved, resulting in redundancy of inertial sensing data, thereby ensuring data accuracy and improving the accuracy of the subsequent data fusion.

In some embodiments, when there are a plurality of inertial sensors of each type, the average value or median of the inertial sensor data obtained through the inertial sensors of a same type may be used for the data fusion. For example, for the angular velocity data of the driving device, the average value or median of the angular velocity data of the driving device obtained through the inertial sensors of a same type may be used as the angular velocity data of the driving device for the data fusion. Data obtained through the inertial sensors of other types may be processed similarly.

In some embodiments, working abnormality includes: one or more of the pitching angle, the rolling angle, and the yawing angle are out of synchronization. In some embodiments, the working abnormality may also include: the parallel mechanism including one or more fractured pre-tensioned metal strips or steel belts.

In some embodiments, the method may further include: when it is determined that the robot apparatus is working abnormally, issuing an alarm. In some embodiments, the robotic apparatus may be stopped from working after issuing the alarm. It can be seen that the alarm issued when the robotic apparatus is working abnormally can facilitate users to stop and repair timely.

In some embodiments, the alarm may be in sound and light mode, or may be display, on a display device of the robotic apparatus, of contents indicating non-synchronization of one or more of the pitching angle, rolling angle, and yawing angle, and/or fracture of the metal strips or steel belts in the transmission device.

In some embodiments, determining whether one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle are synchronized with the one or more respective target angles, includes:

- determining whether each of the actual pitching angle, the actual rolling angle or the actual yawing angle falls within a respective preset angle range, where the respective preset angle range includes a respective target angle of the one or more respective target angles; and

in response to one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle not falling within the respective preset angle range, determining that the actual pitching angle, the actual rolling angle or the actual yawing angle is not synchronized with the one or more respective target angles.

In some embodiments, the preset angle range may be a range centered at the target angle. For example, when the target angle is A_t, the preset angle range may be [A_t−a, A_t+b], where a and b may be the same or different, and a and b are positive numbers. It should be understood that each of the pitching angle, the rolling angle, and the yawing angle has a respective target angle.

The values of a and b may be set as needed and are not limited in the present disclosure. For example, the values of a and b may be set considering the transitional bumps caused by the structural flexibility of the transmission device.

In some embodiments, the target angle may be the angle output by the force providing device, for example, one or more of the pitching angle, the rolling angle, and the yawing angle output by the compact motor drive.

In some embodiments, the operation S430 of determining the operation status of the robotic apparatus based on the fused data, further includes:

- in response to the actual pitching angle, the actual rolling angle and the actual yawing angle each falling within the respective preset angle range, determining that each of the actual pitching angle, the actual rolling angle and the actual yawing angle is synchronized with a respective target angle.

In some embodiments, the operation S430 of determining the operation status of the robotic apparatus based on the fused data, further includes:

- in response to one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle not falling within the respective preset angle range, determining that the actual pitching angle, the actual rolling angle or the actual yawing angle is not synchronized with the one or more respective target angles.

In some embodiments, when each of the actual pitching angle, the actual rolling angle and the actual yawing angle is synchronized with a respective target angle, it is determined that the robotic apparatus is working normally.

In some embodiments, the operation S430 of determining the operation status of the robotic apparatus based on the fused data, and controlling the robotic apparatus in response to the operation status, further includes:

- calculating one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle based on one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle and the one or more respective target angles;
- determining whether each synchronization rate of the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle is greater than or equal to a respective synchronization rate threshold; and
- in response to at least one synchronization rate of the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle being less than one or more respective synchronization rate thresholds, determining that the robotic apparatus is working abnormally and controlling the robotic apparatus to stop working.

In some embodiments, the operation S430 of determining the operation status of the robotic apparatus based on the fused data, and controlling the robotic apparatus in response to the operation status, further includes:

- calculating, based on the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle, one or more respective changes in the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle within a first preset time period, and determining whether the one or more respective changes exceed one or more respective change thresholds; and
- in response to the one or more respective changes in the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle within the first preset time period exceeding the one or more respective change thresholds, determining that the robotic apparatus is working abnormally and controlling the robotic apparatus to stop working.

The one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle of the driving device or the one or more respective changes in the one or more respective synchronization rates may be used to determine whether the robotic apparatus is working normally. In response to the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle being less than one or more respective synchronization rate thresholds, or in response to the one or more respective changes in the one or more respective synchronization rates for one or more of the actual pitching angle, the actual rolling angle and the actual yawing angle being excessive (being greater than the one or more respective change thresholds), it is determined that the robotic apparatus is working abnormally, and the robotic apparatus may be controlled to stop working, in order to prevent the robotic apparatus from being damaged.

It can be seen that in the method for controlling the robotic apparatus according to the embodiments of the present disclosure, the rolling angles, the pitching angles, and the yawing angles of the transmission device (such as a parallel mechanism) and the driving device (such as an insertion arm for surgical instrument) are detected concurrently, thereby achieving multi-directional movement detection of the robotic apparatus and increasing the comprehensiveness of angle synchronization monitoring safety system for the surgical robot.

In some embodiments, the operation S420 of performing the data fusion on at least the inertial sensing data to obtain the fused data of the robotic apparatus, includes:

- acquiring a control signal of the robotic apparatus; and
- fusing the inertial sensing data and the control signal to obtain the fused data.

By fusing the control signals of the robotic apparatus with the inertial sensing data detected by the inertial sensor, the obtained fused data contains more comprehensive and accurate information. The state analysis of the robotic apparatus is performed based on the more comprehensive and accurate fused data, in this way, accuracy of the state analysis of the robotic apparatus can be improved, thereby facilitating accurate control of the robotic apparatus.

In some embodiments, the control signal of the robotic apparatus may include: an output signal of the force providing device (i.e. an input signal of the transmission device), such as output angle, output current, output torque or the like; or an output signal of the driving device, such as driving torque output or the like. It should be understood that the control signal may be any signal that controls the devices or components in the robotic apparatus, and is not limited in the present disclosure.

In some embodiments, the robotic apparatus further includes a transmission device connected to the driving device and configured to move the driving device.

The operation S420 of performing the data fusion on at least the inertial sensing data to obtain the fused data of the robotic apparatus, includes:

- calculating an output angle of the transmission device based on the inertial sensing data;
- calculating, each time, a respective difference between the output angle of the transmission device and an input angle of the transmission device; and
- estimating a permanent consumption of the transmission device based on a mathematical kinematic model for the transmission device and one or both of a change and a rate of change in neighboring differences.

The operation S430 of determining the operation status of the robotic apparatus based on the fused data, includes:

determining, based on the permanent consumption, whether the robotic apparatus is working normally, and determining remaining usage time or a remaining usage number of times of the transmission device.

Before the operation of calculating, each time, a respective difference between the output angle of the transmission device and the input angle of the transmission device, the method may further include: acquiring the input angle of the transmission device.

The input angle of the transmission device is the output angle of the power device, and the power device can be controlled by the control device. That is, the control device sends corresponding control commands to the power device to control the output angle of the power device, and the output angle of the power device (i.e. the input angle of the transmission device) can be used as the control signal.

In some embodiments, the operation of determining, based on the permanent consumption, whether the robotic apparatus is working normally, and determining remaining usage time or a remaining usage number of times of the transmission device, includes:

- when the permanent consumption exceeds a preset consumption, determining that the robotic apparatus is working abnormally, and the remaining usage time of the transmission device is 0.

In some embodiments, when it is determined that the robotic apparatus is working abnormally, the robotic apparatus is controlled to stop working.

In some embodiments, the operation of determining, based on the permanent consumption, whether the robotic apparatus is working normally, and determining remaining usage time or a remaining usage number of times of the transmission device, includes:

- when the permanent consumption does not exceed the preset consumption, determining that the robotic apparatus is working normally, and the remaining usage time or the remaining usage number of times of the transmission device is obtained by dividing the difference between the permanent consumption and the preset consumption by an average consumption.

In some embodiments, the average consumption may be calculated based on the permanent consumption and the usage time or the usage number of times corresponding to the permanent consumption. For example, average consumption may be obtained by dividing permanent consumption by corresponding usage time, or average consumption may be obtained by dividing permanent consumption by corresponding usage number of times.

In some embodiments, the preset consumption may be the consumption corresponding to the time point at which pre-tensioning or replacement is required. When the preset consumption indicates the time point at which pre-tensioning is required, the calculated remaining usage time or remaining usage number of times of the transmission device indicates the remaining usage time or remaining usage number of times before the next pre-tensioning of the transmission device. When the preset consumption indicates the time point at which replacement is required, the calculated remaining usage time or remaining usage number of times of the transmission device indicates the remaining usage time or remaining usage number of times before replacement of the transmission device. It should be understood that the preset consumption may be set as needed, and is not limited in the present disclosure.

In some embodiments, the input angle of the transmission device may be the output angle of the force providing device, for example, the pitching angle, rolling angle, and yawing angle output by the compact motor drive.

The transmission device may be a parallel mechanism, and a mathematical kinematic model for the transmission device may be the mathematical kinematic model for the parallel mechanism. The permanent consumption of the transmission device may be estimated based on the mathematical kinematic model for the transmission device and one or both of a change and a rate of change in neighboring differences between the respective inputs and the respective outputs of the transmission device, in order to obtain the remaining usage time of the transmission device. The permanent consumption may refer to the unrecoverable consumption in the transmission device. Taking a transmission strip in the transmission device as an example, the permanent consumption refers to the unrecoverable elongation of the transmission strip, and the remaining usage number of times or the remaining usage time of the transmission strip may be estimated based on the elongation of the transmission strip in each use of the transmission device.

In some embodiments, the method 400 for controlling the robotic apparatus may further include:

- reporting, by an output device of the robotic apparatus, one or more of the permanent consumption, remaining usage time, and remaining usage number of times of the transmission device.

In some embodiments, the method 400 for controlling the robotic apparatus may further include:

- determining the time point at which maintenance of the transmission device is required based on one or both of the remaining usage time and the remaining usage number of times of the transmission device.

In some embodiments, the method 400 for controlling the robotic apparatus may further include:

- reporting, by the output device of the robotic apparatus, a maintenance duration for the transmission device.

It can be seen that by estimating one or more of the permanent consumption, the remaining usage time, and the remaining usage number of times of the transmission device, maintenance work can be reasonably arranged, thereby reducing maintenance times and costs, and improving maintenance efficiency.