CN118802989B

CN118802989B - Bandwidth adjustment method, system and storage medium of remote surgical robot

Info

Publication number: CN118802989B
Application number: CN202411290891.9A
Authority: CN
Inventors: 王建辰; 高元倩; 张旭; 马鑫; 蒋成军; 万强
Original assignee: Shenzhen Edge Medical Co Ltd
Current assignee: Shenzhen Edge Medical Co Ltd
Priority date: 2024-09-14
Filing date: 2024-09-14
Publication date: 2025-01-28
Anticipated expiration: 2044-09-14
Also published as: CN118802989A

Abstract

The present application discloses a bandwidth adjustment method, system and storage medium of a remote surgical robot, including: a remote surgical robot, including a first terminal and a second terminal in communication connection, one of which is a remote doctor console and the other is a patient surgical platform; a central server, which is in communication connection with at least one of the first terminal and the second terminal; the bandwidth adjustment method includes: in response to the interactive operation of the first terminal to the second terminal, triggering the first control state of the remote surgical robot; according to the first control state, adjusting the bandwidth of the communication connection between the first terminal and the second terminal through the central server. The present invention realizes the adaptive dynamic adjustment of the required network bandwidth for different operations of the remote surgical robot during the remote surgery execution process.

Description

Bandwidth adjustment method, system and storage medium of tele-operation robot

Technical Field

The invention belongs to the field of medical equipment, and particularly relates to a bandwidth adjusting method, a system and a storage medium of a tele-operation robot.

Background

With the continuous perfection of teleoperation robots and network infrastructure, the adoption of the operation robots for teleoperation has become a mature operation technology of hospitals, the acceptance of patients for teleoperation is also increasing, and the demands of markets for the teleoperation robots are also increasing. Tele-surgery is highly dependent on network communication, and network delays and instabilities can directly affect the quality and effectiveness of the surgery.

On the other hand, the network bandwidth of the tele-surgical robot is usually required to meet a certain size, for example, above 60Mbps, and especially in application scenarios such as remote diagnosis and surgery, the requirement on real-time performance is very strict, and a higher bandwidth may be required. The network bandwidth requirements are not the same for different tele-surgical scenarios, but the existing tele-surgery cannot adaptively and dynamically adjust the network bandwidth required for real-time tele-surgical scenarios based on the operations in the procedure execution.

Disclosure of Invention

In order to solve the problem that the existing remote operation can not be based on the operation in the operation execution process, the invention provides the following technical scheme:

The invention provides a bandwidth adjusting method of a tele-operation robot, which comprises the steps of the tele-operation robot comprising a first terminal and a second terminal which are in communication connection, wherein one end of the first terminal and one end of the second terminal are a tele-doctor console, the other end of the first terminal and one end of the second terminal are patient operation platforms, a central server is in communication connection with at least one end of the first terminal and one end of the second terminal, the bandwidth adjusting method comprises the steps of responding to interactive operation of the first terminal on the second terminal, triggering a first control state of the tele-operation robot, and adjusting the bandwidth of the communication connection between the first terminal and the second terminal through the central server according to the first control state.

The method comprises the steps of matching bandwidth parameters corresponding to a first control state through at least one end of the first terminal and the second terminal, sending a first bandwidth adjustment request to the center server based on the bandwidth parameters, and initiating a second bandwidth adjustment request to the network operator server based on the first bandwidth adjustment request through the center server so as to adjust the bandwidth of the communication connection between the first terminal and the second terminal through the network operator server.

Preferably, at least one end of the first terminal and the second terminal stores mapping information, the bandwidth parameter comprises a plurality of bandwidth levels according to a bandwidth high-low sequence, the mapping information comprises a plurality of different mapping relations between the first control states and the bandwidth levels, and a first bandwidth adjustment request is sent to the central server based on the bandwidth parameter, wherein the first bandwidth adjustment request is sent to the central server based on the bandwidth parameter, and comprises a target bandwidth level corresponding to the first control states is matched in the stored mapping information through at least one end of the first terminal and the second terminal, and the first bandwidth adjustment request is sent to the central server based on the target bandwidth level.

Preferably, the interactive operation comprises an operation for triggering any one of a plurality of operation phases and an operation for triggering any one of a plurality of operation tasks, and the first control state represents a state of interactive operation authority of the first terminal for the second terminal in any one operation phase and/or any one operation task.

Preferably, the optional operation stage at least comprises any one of a start-up self-check, remote pairing, remote connection, remote interaction communication, remote operation, remote pairing release and shutdown self-check, and the optional operation task at least comprises any one of an operation task, a teaching task and a debugging task.

Preferably, the tele-doctor console and the patient operating platform comprise operating components, wherein the operating components comprise at least one of a graphical user interface and physical hardware, wherein the triggering of the first control state of the tele-surgery robot in response to the interactive operation of the first terminal on the second terminal comprises the triggering of the interactive operation of the patient operating platform in response to the touch operation of the graphical user interface of the tele-doctor console or the physical operation of the physical hardware, and the triggering of the interactive operation of the tele-doctor console in response to the touch operation of the graphical user interface of the patient operating platform or the physical operation of the physical hardware.

Preferably, the remote operation robot comprises a first terminal, a second terminal, a central server, at least one auxiliary operation device, at least one communication device, at least one remote operation robot and at least one detachable connection device, wherein at least one end of the first terminal and at least one auxiliary operation device is detachably connected to at least one end of the second terminal, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the first control state, the communication device comprises a second control state of the remote operation robot, the second control state of the remote operation robot is triggered in response to the interaction operation executed by the first terminal or the second terminal and the auxiliary operation device, the interaction operation at least comprises the dismounting operation corresponding to the detachable connection, when the first control state and the second control state are triggered at the same time, one control state is selected from the first control state and the second control state according to the preset control state priority, and the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the selected control state.

Preferably, the remote operation robot further comprises an in-vitro image acquisition device which is in communication connection with a first terminal or a second terminal detachably connected with the auxiliary operation device, and before the second control state of the remote operation robot is triggered in response to the interaction operation of the first terminal or the second terminal with the auxiliary operation device, the remote operation robot further comprises the step of acquiring an operation scene image through the in-vitro image acquisition device, wherein the operation scene image comprises the auxiliary operation device and an image of the first terminal or the second terminal detachably connected with the auxiliary operation device, and the dismounting operation of the first terminal or the second terminal with the auxiliary operation device is identified through the first terminal or the second terminal detachably connected with the auxiliary operation device according to the operation scene image.

Preferably, the first control state comprises a third control state of the remote doctor control platform for the patient operation platform and a fourth control state of the patient operation platform for the remote doctor control platform, wherein a master-slave state priority is preset between the third control state and the fourth control state, the center server is in communication connection with one end of the remote doctor control platform and one end of the patient operation platform, the bandwidth of the communication connection between the first terminal and the second terminal is regulated through the center server according to the first control state, the third control state is sent to the patient operation platform through the remote doctor control platform, one of the current fourth control state and the received third control state is selected by the patient operation platform according to the master-slave state priority, or the fourth control state is sent to the remote doctor control platform through the patient operation platform, the bandwidth of the communication connection between the remote doctor control platform and the patient operation platform is selected by the remote doctor control platform according to the master-slave state priority, the one of the current third control state and the fourth control state is selected by the remote doctor control platform.

Preferably, the first control state comprises a third control state of the remote doctor control platform for the patient operation platform and a fourth control state of the patient operation platform for the remote doctor control platform, wherein master-slave state priorities are preset between the third control state and the fourth control state, the central server is respectively in communication connection with the remote doctor control platform and the patient operation platform, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the first control state, the bandwidth of the communication connection between the remote doctor control platform and the patient operation platform is adjusted according to the selected control state, and the bandwidth of the communication connection between the remote doctor control platform and the patient operation platform is adjusted.

The second aspect of the invention provides a bandwidth adjusting system of a tele-surgical robot, which comprises the tele-surgical robot, a central server, a processor and a control system, wherein the tele-surgical robot comprises a first terminal and a second terminal which are in communication connection, one end of the first terminal and one end of the second terminal are a tele-doctor console, the other end of the first terminal and one end of the second terminal are patient operation platforms, the central server is in communication connection with at least one end of the first terminal and one end of the second terminal, the processor is configured to trigger a first control state of the tele-surgical robot in response to the interaction operation of the first terminal on the second terminal, and the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the first control state.

A third aspect of the present invention provides a readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the above method of bandwidth adjustment of a tele-surgical robot.

The remote operation system has the beneficial effects that the patient operation platform and the remote doctor control console in the remote operation robot trigger corresponding control states in the interactive operation process, and represent remote operation scenes such as different operation stages, different operation tasks, different operation types and the like in the operation execution process, so that the network bandwidth required by the real-time remote operation scene can be adaptively and dynamically adjusted according to different operations (different remote operation scenes) of the remote operation robot in the operation execution process.

Drawings

FIG. 1 is a system schematic diagram of an embodiment of a tele-surgical robot of the present invention;

FIG. 2 is a schematic view of the structure of an embodiment of the surgical instrument of the present invention;

FIG. 3 is a system schematic diagram of another embodiment of a tele-surgical robot of the present invention;

FIG. 4 is a first timing diagram of the network communication bandwidth adjustment of the present invention;

FIG. 5 is a second timing diagram of the network communication bandwidth adjustment of the present invention;

FIG. 6 is a schematic diagram of bandwidth allocation for multiple groups of tele-surgical robots of the present invention;

FIG. 7 is a third timing diagram of the network communication bandwidth adjustment of the present invention;

Fig. 8 is a schematic diagram of a first flow of the bandwidth adjustment method of the present invention;

FIG. 9 is a schematic structural view of the tele-surgical robot interoperation of the present invention;

FIG. 10 is a simplified schematic illustration of the connection of the tele-surgical robot and auxiliary surgical device of the present invention;

FIG. 11 is a schematic diagram of a preferred embodiment of the bandwidth adjustment method of the present invention;

FIG. 12 is a fourth timing diagram of the network communication bandwidth adjustment of the present invention;

Fig. 13 is a second flow chart of the bandwidth adjustment method of the present invention;

FIG. 14 is a fifth timing diagram of the network communication bandwidth adjustment of the present invention;

fig. 15 is a schematic diagram of a third flow chart of the bandwidth adjustment method of the present invention;

FIG. 16 is a sixth timing diagram of the network communication bandwidth adjustment of the present invention;

fig. 17 is a schematic flow chart of the bandwidth allocation method of the present invention.

Detailed Description

In order that the invention may be readily understood, a more particular description thereof will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

It should be noted that all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs unless defined otherwise. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. For example, the term "plurality" includes two and more.

1. Tele-operation robot

The remote operation technology of the robot refers to that an information network channel is used as a transmission carrier of medical information, a master-slave method is adopted, and the combination of the robot technology, a virtual reality technology, an artificial intelligence technology, a computer technology and the like is expanded to be used for remote diagnosis, remote guidance, remote operation and the like of medical operation, so that a remote doctor can control an operation robot through the information network channel to perform operation on a remote patient. The remote operation robot comprises master and slave devices, wherein the master operation device is a remote doctor control console, the slave operation device is a patient operation platform, and a doctor performs operation on the remote doctor control console and sends a control command to the slave operation device so as to control the patient operation platform to perform operation on a patient.

Because the first geographic location and the second geographic location are generally remote from each other, such as in different hospitals, buildings, areas, countries, etc., the remote doctor console and the patient operating platform transmit operating procedure data, such as master-slave control signaling, endoscope video data, audio data, haptic feedback data, and interactive data of master-slave devices, such as pairing data, etc., through the information network channel.

Specifically, the remote doctor control console is provided with a first remote host, the patient operation platform is provided with a second remote host, and the remote doctor control console and the patient operation platform are connected to the information network channel through the first remote host and the second remote host respectively, so that transmission of operation process data is realized.

The tele-surgical robots may include, in particular, endoscopic surgical robots (single-hole, multi-hole), trans-natural-cavity surgical robots, percutaneous aspiration surgical robots, orthopedic surgical robots, pan-vascular surgical robots, dental surgical robots, neurosurgical robots, etc., surgical robots including the aforementioned tele-doctor console and patient surgical platform, so that tele-surgical techniques may be implemented.

Illustratively, as shown in fig. 1 and 2, the laparoscopic surgical robot (single bore) includes a tele-doctor console 100 and a patient surgical platform 200. Wherein, the remote doctor console 100 is used for sending control commands to the patient operation platform 200 according to the operation of a doctor to control the patient operation platform 200, and is also used for displaying images acquired by the patient operation platform 200. The patient operating platform 200 is used for responding to control commands sent by the remote doctor console 100 and performing corresponding operations, and the patient operating platform 200 is also used for acquiring in-vivo images.

Patient surgical platform 200 includes a robotic arm 210, a power mechanism 220 disposed on robotic arm 210, a surgical instrument 230 disposed on power mechanism 220, and a cannula 240 that houses a long shaft 231 of surgical instrument 230. In response to control commands from the remote doctor console 100, the patient operating platform 200 is configured to adjust the position of the surgical instrument 230, the power mechanism 220 is configured to drive the surgical instrument 230 to perform a corresponding operation, and the end effector 232 of the surgical instrument 230 is configured to extend into the body and perform a surgical procedure with its distally located end instrument, and/or to acquire in-vivo images.

By way of example, fig. 3 illustrates a natural orifice surgical robot 300. The trans-natural orifice surgical robot 300 includes a tele-doctor console and a patient surgical platform 320, the tele-doctor console including a handle 310 and an imaging cart 330 that are interconnected, and/or the patient surgical platform 320 also includes an imaging cart 330. The patient table 320 has attached thereto a catheter instrument 340 and a sensor system 350, a control system 360 for effecting control among the catheter instrument 340, the sensor system 350, and the imaging cart 330, etc. When a doctor performs various procedures on a patient beside the patient operation platform 320, the doctor can trigger control instructions by operating the handle 310 and send the control instructions to the patient operation platform 320 to drive, so as to control the advancing, retracting, bending and steering of the catheter apparatus 340.

The patient table 320 may be generally moved to the side of the operating table for engaging the catheter instrument 340 and controlling the catheter instrument 340 to be raised and lowered in a vertical direction or to be translated in a horizontal direction or to be moved in a non-vertical and non-horizontal direction under control instructions to provide a better pre-operative preparation angle for the operation of the catheter instrument 340. The control instruction may be an instruction triggered by the doctor by operating the patient operation platform 320, or an instruction triggered by the doctor directly clicking or pressing a button provided on the patient operation platform 320. Of course, in other embodiments, the control command may also be a voice control or a command triggered by a force feedback mechanism.

As shown in fig. 3, further, the patient operation platform 320 may include a base 321, a sliding base 322 capable of moving up and down along the base 321, and 2 mechanical arms 323 fixedly connected with the sliding base 322. The robotic arm 323 may include a plurality of arm segments coupled at joints that provide the robotic arm 323 with a plurality of degrees of freedom, e.g., seven degrees of freedom corresponding to seven arm segments. The distal end of the mechanical arm 323 is provided with a power part (not shown in the figure), and the power part of the mechanical arm 323 is used for engaging the catheter apparatus 340 and controlling the distal end of the catheter apparatus 340 to bend and turn correspondingly under the driving action of the power part. Wherein the 2 mechanical arms 323 may be of identical or partially identical construction, one mechanical arm 323 being adapted to engage the inner catheter instrument 341 and the other mechanical arm 323 being adapted to engage the outer catheter instrument 342. When the outer catheter device 342 is installed, the catheter of the inner catheter device 341 may be inserted into the catheter of the outer catheter device 342 after the outer catheter device 342 is installed.

The sensor system 350 has one or more subsystems for receiving information about the catheter instrument 340. The subsystems may include a position sensor system, a shape sensor system for determining the position, orientation, speed, velocity, pose, and/or shape of the tip of the catheter instrument 340 and/or along one or more sections of a catheter that may constitute the catheter instrument 340, and/or a visualization system for capturing images from the tip of the catheter instrument 340.

The image cart 330 may be provided with a display system 331, a flushing system (not shown), and the like. The display system 331 is used to display images or representations of the surgical site and catheter instrument 340 generated by the subsystems of the sensor system 350. Real-time images of the surgical site and catheter instrument 340 captured by the visualization system may also be displayed. Image data from imaging techniques such as Computed Tomography (CT), magnetic Resonance Imaging (MRI), optical Coherence Tomography (OCT), ultrasound, and the like may also be used to present images of the surgical site recorded preoperatively or intraoperatively.

Wherein the pre-or intra-operative image data may be presented as two-, three-or four-dimensional (e.g. time-based or rate-based information) images and/or as images from a model created from the pre-or intra-operative image dataset, virtual navigation images may also be displayed. In the virtual navigation image, the actual position of the catheter instrument 340 is registered with the pre-operative image to externally present a virtual image of the catheter instrument 340 within the surgical site to the operator.

The control system 360 includes at least one memory and at least one processor. It is understood that the control system 360 may be integrated into the patient table 320 or the imaging cart 330, or may be provided separately. The control system 360 may support wireless communication protocols such as IEEE 802.11, irDA, bluetooth, homeRF, DECT, and wireless telemetry, among others. The control system 360 may transmit one or more signals indicative of movement of the catheter instrument 340 by the power section to move the catheter instrument 340. The catheter apparatus 340 may extend to a surgical site within the body via an opening or surgical incision of a natural orifice of the patient.

Further, the control system 360 may include a mechanical control system (not shown) for controlling movement of the catheter apparatus 340 and an image processing system (not shown), and thus may be integrated into the patient surgical platform 320. The image processing system is used for virtual navigation path planning and thus can be integrated in the video cart 330. Of course, the subsystems of the control system 360 are not limited to the specific cases listed above, and may be reasonably configured according to practical situations.

The image processing system can image the surgical site by using the imaging technology based on the image of the surgical site recorded before or during the operation. Software used in conjunction with manual input may also convert the recorded images into two-or three-dimensional composite images of portions or whole anatomical organs or segments. During the virtual navigation procedure, the sensor system 350 may be used to calculate the position of the catheter instrument 340 relative to the patient's anatomy, which may be used to generate external tracking images and internal virtual images of the patient's anatomy, enabling registration of the actual position of the catheter instrument 340 with the preoperative images so that the virtual image of the catheter instrument 340 within the surgical site may be presented to the operator from the outside.

The inner catheter device 341 and the outer catheter device 342 are generally identical in structure composition and have an inner catheter 41 and an outer catheter 42, respectively, which are elongated and flexible, wherein the outer catheter 42 has a diameter slightly larger than the inner catheter 41 so that the inner catheter 41 can pass through the outer catheter 42 and provide a certain support for the inner catheter 41 so that the inner catheter 41 can reach a target site in a patient for tissue or cell sampling from the target site.

Certain movements of the handle 310 may cause corresponding movements of the catheter instrument 340. For example, movement of the steering lever of the handle 310 may be mapped to a corresponding pitching movement of the distal end of the catheter instrument 340 when the steering lever of the handle 310 is operated by the physician, and movement of the steering lever of the handle 310 may be mapped to a corresponding yawing movement of the distal end of the catheter instrument 340 when the steering lever of the handle 310 is operated by the physician to the left or right. In this embodiment, the handle 310 may control the movement of the distal end of the catheter instrument 340 over a 360 ° spatial range.

The teledoctor console and patient operating table are equipped with various auxiliary surgical devices such as the aforementioned surgical instruments, catheter instruments, irrigation systems, operating tables, and others, as well as a stab card, energy table, monitor, fluid aspiration system, sterile mask, pneumoperitoneum machine, etc., in addition to their own equipment. The patient surgical platform is configured with at least a majority of auxiliary surgical devices to directly act on the patient, and the tele-doctor console is configured with a minority of auxiliary surgical devices to assist the doctor in performing the surgical procedure.

2. Network communication bandwidth for tele-surgical robots

The network channel between the remote doctor control console and the patient operation platform is mainly used for representing nodes of the remote doctor control console and the patient operation platform through cloud networking and communicating through a single virtual channel/internet channel. Specifically, an application program may be used to reserve and adjust the bandwidth, please refer to fig. 4, and the specific flow is as follows:

1. Application login

The intelligent terminal device, such as a mobile phone, a computer, a tablet personal computer, a remote doctor control console, a patient operation platform and the like, logs in the central server through an application program. After the login is successful, a resource list is requested to a central server to inquire the resource list of the nodes of the remote doctor control console and the patient operation platform and determine the reservation list state of the nodes, and the nodes of the remote doctor control console and the patient operation platform which can control the bandwidth in the login account authority can be specifically included. The central server receives the request and then requests authentication and resource inquiry and reservation list state from the operator server. After the authentication of the operator server is successful, the resource list and the reservation list state are returned, and the resource list and the reservation list state are forwarded to the application program by the center server.

2. Querying node bandwidth

The resource list and the reservation list state received by the application program are displayed on a display of the intelligent terminal device, at least one node in the resource list is selected on the display, and the inquiry bandwidth is requested to the central server. And after the authentication of the operator server is successful, the node bandwidth is returned, and the node bandwidth is forwarded to an application program by the center server and is displayed on a display of the intelligent terminal device. And determining the node bandwidth size between the corresponding remote doctor control console and the patient operation platform according to the node bandwidth size.

3. Reservation of bandwidth

The resource list and the reservation list state received by the application program are displayed on a display of the intelligent terminal device, at least one node in the resource list is selected on the display, adjustment conditions such as reservation time, reservation bandwidth value, reservation inquiry period and the like are set, and a 'reservation bandwidth node' is initiated to the central server. When the set reservation time is reached, the central server requests the operator server to adjust the selected node to the reserved bandwidth value, so that the network channel between the corresponding remote doctor control console and the patient operation platform maintains the bandwidth of the reserved bandwidth value.

In the process of adjusting the bandwidth by the operator server, the center server periodically inquires the reserved node bandwidth according to a preset inquiry period or a fixed inquiry period (such as every 30 seconds), if the reserved bandwidth value is reached within the preset time, the reserved time of the selected node is cleared, and if the reserved time is not reached within the preset time, a reserved failure prompt is initiated to the application program.

3. Operation and communication of tele-surgical robots

The method comprises the steps that a doctor operates on a control structure such as a touch screen, a pedal, a handle and a button of a remote doctor control console to trigger interactive operation on a patient operation platform, the doctor operates on the control structure such as the touch screen and the button of the patient operation platform to trigger interactive operation on the remote doctor control console, different interactive operations trigger different control states between the remote doctor control console and the patient operation platform, so that data transmission amounts of network channels between the remote doctor control console and the patient operation platform are different, and therefore the network channels need different bandwidths to ensure that network low-delay and data transmission are stable.

Specific operations include power-on self-checking, remote pairing, remote connection, remote interactive communication, remote operation, remote pairing release, shutdown self-checking and the like, or operations such as operation tasks, teaching tasks, debugging tasks and the like, for realizing different operation tasks. Under the corresponding control state of different operation triggers, the data transmission quantity of the network channel is different, and the data types transmitted in the network channel are different.

Illustratively, at least equipment information data, authentication data, protocol data and the like are transmitted in the network channel during remote pairing, remote connection and remote pairing release, at least video data and/or audio data and the like are transmitted in the network channel during remote interactive communication and teaching tasks, and at least video data, audio data and/or control signaling data and the like are transmitted in the network channel during remote operation and operation tasks. The data type of the interactive data packet can be specifically identified by designating a flag bit in the interactive data packet.

Further, the corresponding bandwidth is adjusted according to the type of data transmitted in the network channel, for example, a bandwidth of 1Mbps is allocated when only control signaling data is transmitted, a bandwidth of 10Mbps is allocated when teaching task related data such as video data and/or audio data is transmitted, and a bandwidth of 100Mbps is allocated when surgical operation task related data such as video data, audio data and/or control signaling data is transmitted.

For the operation and communication of the tele-surgical robot, bandwidths of different surgical phases, surgical tasks and/or surgical types can be preset, and the bandwidth size can be automatically requested to be adjusted when triggered without manually reserving and adjusting the bandwidth size by using an application program. As shown in FIG. 5, an embodiment of automatically adjusting bandwidth is provided, for example, the bandwidth of the remote pairing between the remote doctor console and the patient operation platform is preset to be 10Mbps, and the flow of the remote pairing and the bandwidth adjustment between the remote doctor console and the patient operation platform is described. The specific description is as follows:

The method comprises the steps of connecting a remote doctor control console and a patient operation platform to a central server, establishing communication connection with the central server respectively, and initiating registration to the central server based on the communication connection, wherein the registration information comprises IP information, port information, network type and the like of each equipment board card. After registration is completed, at least one end logs in the central server to finish online, and the online state is maintained.

And (3) remote pairing, namely respectively initiating pairing requests to a central server by the remote doctor control console and the patient operation platform, and after the two ends of the remote doctor control console and the patient operation platform are paired successfully by the central server, connecting the communication connection of the remote doctor control console and the patient operation platform and returning corresponding network information.

The method comprises the steps of adjusting bandwidth, setting preset bandwidth of a network channel in a remote pairing stage, namely 10Mbps before switching to other operation stages, operation tasks and/or operation types, requesting the adjustment bandwidth to 10Mbps by a remote doctor control console, initiating a request for inquiring the bandwidth to an operator server after the request is received by the center server, authenticating and requesting the operator server to adjust the bandwidth value by 10Mbps after receiving confirmation inquiry information, and forwarding the bandwidth returned by the operator to the remote doctor control console after adjustment is completed. Wherein requesting to the central server to adjust the bandwidth to 10Mbps may also be a patient surgical platform.

In particular, the stage of the procedure, the task of the procedure, and/or the type of procedure to enter/switch may be determined by at least one of 1) the control state between the tele-doctor console and the patient surgical platform, 2) the connection state of the auxiliary surgical device detachably connected to the tele-doctor console and the patient surgical platform, and 3) the type of data transmitted in the network channel.

In addition, when a plurality of pairs of master-slave devices (a remote doctor control console and a patient operation platform) exist in the same network channel, each pair of master-slave devices share the bandwidth of the same network channel and transmit data based on the bandwidth, on one hand, the data transmission of each pair of master-slave devices needs to be ensured to be low in delay and stable, and on the other hand, the total required bandwidth of each pair of master-slave devices is ensured to be within the bandwidth threshold value of the network channel. It is therefore necessary to dynamically allocate the bandwidth of each pair of master-slave devices based on the operative phase M, the operative task N, and/or the operative type K that each pair of master-slave devices is performing, so as to satisfy the bandwidth limitations of the network channel and the operative requirements of each pair of master-slave devices.

Illustratively, as shown in FIG. 6, the teledoctor console A and the patient surgical platform A are performing a surgical stage m1, a surgical task n1, a surgical type k1 with a corresponding allocated high bandwidth (e.g., 100 Mbps), the teledoctor console C and the patient surgical platform C are performing a surgical stage m3, a surgical task n3, a surgical type k3 with a corresponding allocated high bandwidth (e.g., 60 Mbps), and the teledoctor console B and the patient surgical platform B are performing a surgical stage m2, a surgical task n2, a surgical type k2 with a corresponding allocated low bandwidth (e.g., 10 Mbps).

4. Bandwidth dynamic adjustment based on master-slave device interaction operation process

The bandwidth adjustment system of the tele-surgical robot comprises the tele-surgical robot, a central server, a processor and a control system, wherein the tele-surgical robot comprises a first terminal and a second terminal which are in communication connection, one end of the first terminal and one end of the second terminal are a tele-doctor control console (master hand equipment), the other end of the first terminal and one end of the second terminal are patient operation platforms (slave hand equipment), the central server is in communication connection with at least one end of the first terminal and the second terminal, the processor is configured to trigger a first control state of the tele-surgical robot in response to the interaction operation of the first terminal on the second terminal, and the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the first control state.

In this embodiment, the interactive operation of the first terminal with respect to the second terminal means that, when the first terminal performs an operation, the second terminal performs a control response following the operation of the first terminal, and the operation is the interactive operation. For example, the operation of the operating parts (clamping jaw/finger ring, handle, etc.) of the remote doctor control console is carried out, the mechanical arm and the surgical instrument of the patient operation platform are controlled in a motion mode, the parameters of the endoscope are set on the touch screen on the patient operation platform, and the display screen on the remote doctor control console is controlled in a display mode.

In response to an interactive operation on the first terminal, control information for the second terminal is generated, the first control state is represented based on the control information, or the first control state is mapped to the target in a plurality of preset control states according to the control information. The control information may be information automatically generated when the first terminal is operated, and the first control state is a state variable configured based on bandwidth adjustment.

The method comprises the steps of configuring a state variable of a first control state, configuring a bandwidth size corresponding to each first control state based on bandwidth adjustment, wherein the bandwidth size is a dependent variable of the state variable, matching the corresponding bandwidth size based on the first control state, and requesting a central server to adjust the bandwidth according to the matched bandwidth size.

The central server is used for communicating with the network operator server, carrying out bandwidth adjustment on each group of teleoperation robots logged on the central server, carrying out unified communication with the network operator server based on the bandwidth adjustment request, and adjusting the bandwidth of the network channel to the bandwidth of the request.

In addition, when the bandwidth of the communication connection between the first terminal and the second terminal is adjusted, one end of the bandwidth is in communication connection with the central server, or two ends of the bandwidth are respectively in communication connection with the central server, namely the bandwidth is in communication connection with a remote doctor control console or a patient operation platform, and the bandwidth is in communication connection with the central server. In order to reduce the load of the central server, it is preferable to be communicatively connected to one of the first terminal and the second terminal.

Specifically, fig. 7 illustrates a bandwidth adjustment flow for a first terminal and a second terminal when only the first terminal is communicatively connected to a central server. The second terminal sends a first control state triggered by the second terminal to the first terminal, the first terminal checks the first control states triggered by the first terminal and the second terminal in real time according to the operation process/a preset period, when the first control state is unchanged, bandwidth adjustment is not needed, and when the first control state is changed, bandwidth adjustment is requested to the central server based on the changed first control state.

The central server is triggered to initiate a bandwidth adjustment request to the network operator server after receiving the request, wherein the node bandwidth of the first terminal is firstly inquired to the network operator server and returned according to the confirmation inquiry information, the central server sends the authentication of the central server or the identity information of the first terminal, the authentication is carried out by the operator server, the security of bandwidth adjustment is ensured, the node bandwidth of the first terminal can be adjusted to the requested bandwidth after the authentication is passed, the central server is forwarded to the first terminal after receiving the adjusted bandwidth, and the first terminal can be prompted.

Based on the foregoing bandwidth adjustment system of the tele-surgical robot, a bandwidth adjustment method of the tele-surgical robot is described, please refer to a flow chart of the bandwidth adjustment method of fig. 8, and an embodiment of the bandwidth adjustment method is provided, which is specifically as follows:

801. triggering a first control state of the tele-surgical robot in response to the interactive operation of the first terminal with respect to the second terminal;

In this embodiment, the remote doctor console and the patient operating platform comprise an operating component comprising at least one of a graphical user interface and physical hardware, triggering interactive operations for the patient operating platform in response to touch operations on the graphical user interface of the remote doctor console or physical operations on the physical hardware, and triggering interactive operations for the remote doctor console in response to touch operations on the graphical user interface of the patient operating platform or physical operations on the physical hardware.

Specifically, the graphical user interface of the first terminal is displayed in a display or a touch screen, for example, and comprises at least one control for controlling the second terminal, and the touch operation of the control triggers the interactive operation of the second terminal. Such as a graphical user interface of a remote doctor console, comprises at least one control of a connection control, a pairing control, a robot configuration control (configuration control of an imaging system, an energy platform, a mechanical arm authority and the like), a surgical operation control, a patient surgical platform switching control, a pairing release control, a connection release control and the like for a patient surgical platform. Such as a graphical user interface of a patient operating table, comprises at least one control of a connection control, a pairing control, a robot configuration control (configuration controls of an imaging system, an energy platform, a mechanical arm authority and the like), a remote doctor control console switching control, a pairing release control, a connection release control and the like. Any of the touch operations that may be used to characterize different surgical scenarios may be configured to trigger the first control state, which is not described herein.

The method includes the steps of setting a first control state to include a preoperative control state, an intraoperative control state and a postoperative control state, triggering the preoperative control state in response to touch operation of a connection control, a pairing control and a robot configuration control, triggering the intraoperative control state in response to touch operation of a surgical task selection control and a patient surgical platform switching control, and triggering the postoperative control state in response to touch operation of a pairing release control and a connection release control.

Specifically, among the physical hardware of the first terminal, such as a tele-surgical robot including a controller (e.g., jaw/ring, handle, etc.), foot pedals, actuators, ergonomic components (to adjust the position of the doctor's console for the doctor's operation), control buttons, etc., a patient surgical platform including a tiller, center post, boom pivot, boom, telescopic joint, arm height joint, patient distance joint, robotic arm, base, control buttons, etc.

By way of example, if the first control state is set to include a pre-operative control state, an intra-operative control state, a post-operative control state, a pre-operative control state triggered in response to physical manipulation of an actuator, ergonomic component, on button, mating button, robotic configuration button, or a tiller, center post, boom pivot, boom, telescopic joint, arm height joint, patient distance joint, and mechanical arm of the patient surgical platform, as well as physical manipulation of a base, on button, a post-operative control state triggered in response to physical manipulation of a controller, foot pedal, of the tele-doctor console, a post-operative control state triggered in response to physical manipulation of a shutdown button of the tele-doctor console, a tiller, center post, boom pivot, boom, telescopic joint, arm height joint, patient distance joint, and mechanical arm of the patient surgical platform, and physical manipulation of the shutdown button.

In one embodiment, such as tele-surgical robot 900 shown in FIG. 9, the physical hardware of the tele-doctor console may include a handle 910, a graphical user interface may be provided on the display system 921 on the imaging cart 920, and a corresponding first control state may be triggered in response to physical manipulation of the handle 910 or touch manipulation of the display system 921. The physical hardware of the patient operation platform may include a sliding base 930, and 2 mechanical arms 940 fixedly connected to the sliding base 930, and the graphical user interface may be provided on a display system 951 of an imaging cart 950, and trigger a corresponding first control state in response to a physical operation of the sliding base 930 or the mechanical arms 940, or a touch operation of the display system 951. Any of the interactions that may be used to characterize different surgical scenarios may be configured to trigger the first control state, and are not described in detail herein.

802. And according to the first control state, regulating the bandwidth of the communication connection between the first terminal and the second terminal through the central server.

In this embodiment, the central server is in communication connection with a preset network operator server, and matches bandwidth parameters corresponding to the first control state through at least one end of the first terminal and the second terminal, and sends a first bandwidth adjustment request to the central server based on the bandwidth parameters, and initiates a second bandwidth adjustment request to the network operator server based on the first bandwidth adjustment request through the central server, so as to adjust the bandwidth of the communication connection between the first terminal and the second terminal through the network operator server.

In this embodiment, fig. 7 illustrates that the bandwidth parameter corresponding to the first control state is matched through the first terminal, and the first bandwidth adjustment request is sent to the central server, that is, the bandwidth parameter matching of the first control state of the first terminal and the first bandwidth adjustment request can be respectively executed through the first terminal and the second terminal, and the first bandwidth adjustment request is sent.

In particular, the bandwidth parameters may include specific bandwidth values, such as 10Mbps, 50Mbps, 100Mbps, etc. And can also be a bandwidth numerical range, such as [10Mbps,30Mbps ], [30Mbps,50Mbps ], [50Mbps,70Mbps ], [70Mbps,100Mbps ], etc. The initiated first bandwidth adjustment request includes the bandwidth parameter, and the central server generates a second bandwidth adjustment request based on the bandwidth parameter included in the first bandwidth adjustment request, further requesting the network operator server to adjust the bandwidth.

Further, when the central server inquires that the returned bandwidth of the network operator server is different from the requested bandwidth value, or the returned bandwidth is not in the requested bandwidth value, the network operator server adjusts to the requested bandwidth value. When the bandwidth parameter is a bandwidth value range, a preset value within the bandwidth value range may be selected, and preferably, the bandwidth parameter is a maximum bandwidth value, such as 30Mbps in [10Mbps,30Mbps ].

In one embodiment, the interactive operation comprises an operation for triggering any one of a plurality of operation phases and an operation for triggering any one of a plurality of operation tasks, and the first control state represents a state of interactive operation authority of the first terminal for the second terminal in any one operation phase and/or any one operation task.

At least one end of the first terminal and the second terminal stores a history control state which indicates the current surgical stage and surgical task of the remote surgical robot; the method comprises the steps that at least one end is matched with bandwidth parameters corresponding to a first control state, the first control state and a historical control state are compared, if the comparison result is that the control states are the same, namely, the control states are the same operation stage and operation task, a first bandwidth adjustment request is not required to be sent to a central server, namely, the bandwidth of a network channel is not required to be adjusted, and if the comparison result is that the control states are different, namely, the control states are the operation stage and the operation task are switched, the first bandwidth adjustment request is sent based on the bandwidth parameters of the first control state, and the bandwidth of the network channel is adjusted, so that the operation stage and the operation task after the switching are adapted.

The method comprises the steps of starting up self-checking, remote pairing, remote connection, remote interactive communication, remote operation, remote pairing release and shutdown self-checking, wherein the step of triggering any operation task at least comprises any one of operation task, teaching task and debugging task.

In one embodiment, the surgical phases may further include remote pairing switching, and the partial surgical phase sequencing includes remote pairing, remote connection, remote interactive communication, remote surgical operation/remote pairing release/remote pairing switching, sequencing between surgical phases triggered by the interactive operation and the surgical phases in the history control state, maintaining the bandwidth of the network channel as the latter if the former is before the latter, and adjusting the network channel as the network of the former if the former is after the latter is sequenced.

Each surgical stage and the corresponding interaction operation and interaction operation authority are specifically shown as follows:

A. The first terminal reads the equipment information of the second terminal, verifies the equipment configuration information of the first terminal and the second terminal, and then executes other operation phases on the second terminal. The interactive operation authority at least comprises the starting-up checking authority of the operation.

B. The method comprises the steps of responding to physical operation of a pairing component (such as a button/rocker) on a display screen or touch operation of a pairing control on an image user interface, triggering a remote pairing stage, requesting and displaying a pairing list of a second terminal by a first terminal, initiating a pairing request by a second terminal of a selection target, and completing pairing, wherein the interactive operation authority at least comprises the pairing authority of the operation.

C. The method comprises the steps of responding to physical operation of a configuration component (such as a button/rocker) on a display screen or touch operation of a robot configuration control on a graphic user interface, triggering a remote connection stage, configuring control parameters of an image system, an energy platform, a mechanical arm authority and the like by a first terminal and a second terminal, wherein the interactive operation authority at least comprises the robot configuration authority of the operation.

D. Triggering remote interactive communication in response to physical operation of a communication component (such as a button/rocker/rolling ball) on a display screen or touch operation of a communication control on a graphical user interface; the first terminal and the second terminal can perform selection or operation on image data, voice data, robot states, image labels and the like. The interactive operation authority at least comprises the communication authority of the operation.

E. The method comprises the steps of responding to physical operation of an operation component (clamping jaw/ring, handle and the like) and a pedal on a tele-operation robot or touch operation of an operation control on an image user interface, triggering tele-operation, performing master-slave control, mechanical arm switching and other operations on a patient operation platform by the tele-operation robot, wherein the interactive operation authority at least comprises the operation authority.

F. The method comprises the steps of responding to physical operation of a pairing releasing component (such as a button/rocker) on a display screen or touch operation of a pairing releasing control on a graphical user interface, triggering remote pairing releasing, verifying whether a pairing releasing condition is met by a first terminal and a second terminal (such as whether remote operation is continuously executed or not), and completing a pairing releasing process, wherein the interactive operation authority at least comprises the pairing releasing authority of the operation. The pairing releasing component and the pairing component can be the same component, and different triggering modes of pairing releasing and pairing are set.

G. Triggering a power-on self-test phase in response to physical operation of a power-off component (such as a button/rocker) or touch operation of a power-off control on a graphical user interface; the first terminal and the second terminal verify whether the shutdown condition is met or not, and the shutdown process is completed. The interactive operation authority at least comprises shutdown authority. The starting-up component and the shutdown component can be the same component, and different triggering modes of starting-up and shutdown are set.

H. The method comprises the steps of responding to physical selection operation of a surgical operation task component (such as a button/rocker) or touch operation of a surgical operation task control in a graphical user interface, triggering a surgical operation task, wherein the first terminal and the second terminal can execute surgical operation (such as master-slave control), and the interaction operation authority at least comprises conventional surgical operation authority.

I. the method comprises the steps of responding to physical selection operation of teaching task components (such as buttons/rockers) or touch operation of teaching task controls on a graphical user interface, triggering a teaching task, enabling a first terminal and a second terminal to execute teaching substitution and analysis scene display, and enabling interaction operation permission to at least comprise information transmission permission such as voice/video.

J. The method comprises the steps of responding to physical selection operation of a debugging task component (such as a button/rocker) or touch operation of a debugging task control on a graphic user interface, triggering a debugging task, wherein the first terminal and the second terminal can execute debugging of the tele-surgical robot, and the interactive operation authority at least comprises the tele-surgical robot debugging authority. The surgical operation task component, the teaching task component and the debugging task component can be integrated into one task selection component, and different triggering modes are set.

In addition, a display screen can be arranged to display each operation stage area and one selection control, the physical hardware can be arranged as a direction operation component and a confirmation component, the direction operation component can be 1) two buttons or one rocker which control the selection control to move along the up-down direction or the left-right direction of the display screen, or 2) four buttons or one rocker which control the selection control to move along the up-down direction, the left-right direction of the display screen, and the operation stage is triggered by operating the confirmation component when the selection control stays in a certain operation stage area. The rocker is integrated with the direction operation part and the confirmation part, and can be pressed relative to the vertical direction of the up-down left-right direction of the display screen to confirm triggering the operation stage.

The principle of presetting the bandwidth size for any operation stage and any operation task can be that the bandwidth comprises power on self-check/power off self-check, remote pairing/remote connection/remote pairing release, remote interactive communication/remote operation, operation tasks, debugging tasks and teaching tasks from small to large. The preset bandwidth size may be a bandwidth level, a bandwidth value range, or a bandwidth value.

The method comprises the steps that at least one end of a first terminal and at least one end of a second terminal are stored with mapping information, the bandwidth parameter comprises a plurality of bandwidth levels according to the bandwidth high-low sequence, the mapping information comprises a plurality of different mapping relations between a first control state and the bandwidth levels, a target bandwidth level corresponding to the first control state is matched in the stored mapping information through at least one end of the first terminal and the second terminal, and a first bandwidth adjustment request is initiated to the central server based on the target bandwidth level.

In the embodiment, different bandwidth levels corresponding to the first control state are set according to the preset bandwidth sizes of the operation stage and the operation task, for example, 5 bandwidth levels from low to high, namely 10Mbps, 20Mbps, 30Mbps, 40Mbps and 60Mbps are set, namely { power on self test/power off self test 10Mbps, remote pairing/remote connection/remote pairing release 20Mbps, remote interaction communication/remote operation 60Mbps, operation task 60Mbps, debugging task 40Mbps and teaching task 30Mbps }.

Each bandwidth level corresponds to a bandwidth numerical range or a bandwidth numerical value, the first bandwidth adjustment request comprises the bandwidth numerical range or the bandwidth numerical value, the central server inquires the bandwidth of the network operator server on the current network channel after receiving the first bandwidth request, if the bandwidth level is larger than the bandwidth numerical range, the bandwidth is adjusted to be the upper limit value of the bandwidth numerical range, if the bandwidth level is smaller than the bandwidth numerical range, the bandwidth is adjusted to be the lower limit value of the bandwidth numerical range, and the bandwidth of the current network channel is adjusted to be the requested bandwidth numerical value.

In addition, when the remote doctor control console and the patient operation platform both store mapping information, the remote doctor control console and the patient operation platform are respectively matched with target bandwidth levels of the first control state of the remote doctor control console and the patient operation platform, and respectively initiate first bandwidth adjustment requests to the central server based on the respective target bandwidth levels.

When only the remote doctor control console stores mapping information, the patient operation platform sends the first control state of the remote doctor control console to the remote doctor control console, the remote doctor control console is matched with the target bandwidth level of the first control state of the patient operation platform and the remote doctor control console, and at the moment, the remote doctor control console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

When only the patient operation platform stores mapping information, the remote doctor control platform sends the first control state of the patient operation platform to the patient operation platform, the patient operation platform is matched with the target bandwidth level of the first control state of the remote doctor control platform and the patient operation platform in a unified mode, and at the moment, the patient operation platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

In addition, the mapping information can be stored in the central server, and the central server uniformly matches the target bandwidth level corresponding to the first control state of the remote doctor console and the patient operation platform according to the stored mapping information, and directly initiates a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

In one embodiment, the bandwidth adjusting system further comprises at least one auxiliary operation device, at least one auxiliary operation device is detachably connected to at least one end of the first terminal and at least one end of the second terminal, a second control state of the tele-operation robot is triggered in response to interaction operation executed by the first terminal or the second terminal and the auxiliary operation device, the interaction operation at least comprises dismounting operation corresponding to detachable connection, when the first control state and the second control state are triggered at the same time, one control state is selected from the first control state and the second control state according to a preset control state priority through at least one end of the first terminal and the second terminal, and the bandwidth of communication connection between the first terminal and the second terminal is adjusted through the central server according to the selected control state.

By way of example, as shown in FIG. 10, if an auxiliary surgical device is connected to the remote doctor console, it may include at least one of an endoscope control headset, an electroencephalogram monitoring headset, an eye movement tracking device, a display screen, etc., and if an auxiliary surgical device is connected to the patient surgical platform, it may include at least one of a surgical instrument, a catheter instrument, an irrigation system, a surgical bed, a sterile cover, a punch card, an energy platform, a monitor, a fluid aspiration system, a pneumoperitoneum machine, etc.

For different interactive operations of the first terminal or the second terminal and different auxiliary surgical devices, different surgical stages or surgical tasks are represented, and corresponding second control states are triggered, wherein the second control states comprise a preoperative control state, an intraoperative control state and a postoperative control state, the interactive operations comprise unconnected operations, connected operations and dismantling operations, the unconnected operations correspond to the preoperative control states, the connected operations correspond to the intraoperative control states, and the dismantling operations correspond to the postoperative control states.

Further, when the remote hand robot executes different operation projects, the auxiliary operation devices required by each operation project are preset, the preoperative control state is triggered when all the preset auxiliary operation devices do not complete all the connection operation, the intraoperative control state is triggered when all the connection operation is completed, the postoperative control state is triggered when any one auxiliary operation device is removed, wherein the key auxiliary operation device of each operation project can be further set, and the postoperative control state is triggered when the key auxiliary operation device is removed.

The method includes the steps of setting a first control state of the network channel to be a pre-operation control state, setting the bandwidth of the network channel to be 10Mbps when the first control state is a pre-operation control state, setting the bandwidth of the network channel to be 60Mbps when the first control state is an intra-operation control state, and setting the bandwidth of the network channel to be 10Mbps when the first control state is a post-operation control state;

exemplary critical auxiliary surgical devices for a teledoctor console include endoscope controlled head mounted devices, and critical auxiliary surgical devices for a patient operating platform include surgical instruments, catheter instruments, stab cards, sterile hoods, monitors, and operating tables.

In one embodiment, the preset control state priority may indicate a control state with a lower matched bandwidth value and a higher priority from among the first control state and the second control state, for example, the first control state is a preoperative state, the matched bandwidth is 20Mbps, the second control state is an intraoperative control state, and the matched bandwidth is 60Mbps, which indicates that the tele-surgical robot has not yet started to perform the surgical operation at this time, and the priority of the first control state is higher. Wherein a control state with a higher priority is selected from the first control state and the second control state.

In one embodiment, the preset control state priority may also represent the importance of the interactive operations performed on the teledoctor console and the patient surgical platform relative to the interactive operations performed on the auxiliary surgical device for determining whether bandwidth adjustment or the size of the bandwidth adjustment is required to meet the required bandwidth of the surgical item.

The first control state includes a first preoperative control state (such as power-on self-test, remote pairing, remote connection), a first intra-operative control state (such as remote interactive communication, remote operation), a first post-operative control state (such as remote pairing release, and power-off self-test), the second control state includes a second preoperative control state, a second intra-operative control state, and a second post-operative control state, and the preset control state priority includes, with reference to the first control state:

1. The first preoperative control state is greater than the second intraoperative control state/the second postoperative control state, namely, when the remote doctor control console and the patient operation platform execute the interactive operation of starting self-check, remote pairing and remote connection, the network channel is adjusted to be 10Mbps or 20Mbps, when all auxiliary operation devices complete the connection operation or remove, the operation robot self-check program and the remote pairing program are required to be completed firstly, and the subsequent operation can be executed, so that the bandwidth of the network channel is not required to be adjusted to 60Mbps or 10Mbps;

2. The first intra-operation control state is less than the second pre-operation control state/the second post-operation control state, namely, when the remote doctor control console and the patient operation platform are used for performing remote interactive communication and interactive operation of remote operation, the network channel is adjusted to be 60Mbps, when all auxiliary operation devices are not connected or removed, the operation condition is not met, and the bandwidth of the network channel is only 10Mbps;

3. when the remote doctor control console and the patient operation platform execute the interactive operation of remote pairing release and shutdown self-check, the network channel is adjusted to be 10Mbps or 20Mbps, and the remote operation robot stops the operation, so that even if all auxiliary operation devices complete the connection operation, the operation is not required to be executed, and the network channel is not required to be adjusted to 60Mbps;

in addition, the remote doctor control console and the patient operation platform store historical first control states or second control states which represent the current operation stage and operation task of the remote operation robot, and when the historical first control states and the current second control states are stored or when the historical second control states and the current first control states are stored, the first control states and the second control states are determined to be triggered simultaneously.

The remote operation robot comprises an external image acquisition device, a remote operation robot, an auxiliary operation device, an external image acquisition device, a first terminal or a second terminal which is in communication connection with the auxiliary operation device in a detachable mode, and before a second control state of the remote operation robot is triggered in response to the interaction operation of the first terminal or the second terminal with the auxiliary operation device, the external image acquisition device is used for acquiring an operation scene image, wherein the operation scene image comprises the auxiliary operation device and the image of the first terminal or the second terminal which is in detachable connection with the auxiliary operation device, and the first terminal or the second terminal is identified in detachable connection with the auxiliary operation device according to the operation scene image.

In this embodiment, if the remote doctor console is connected with the auxiliary operation device, an external image acquisition device is arranged at one end of the remote doctor console and an image recognition algorithm is arranged, and if the patient operation platform is connected with the auxiliary operation device, an external image acquisition device is arranged at one end of the patient operation platform and an image recognition algorithm is arranged. The spatial location of the in vitro image acquisition device is set such that the acquisition of the surgical scene image includes the junction of the auxiliary surgical device and the teledoctor console, or includes the junction of the auxiliary surgical device and the patient surgical platform.

A disassembly operation between the remote doctor console and the auxiliary operation device or between the patient operation platform and the auxiliary operation device is identified by applying an image identification algorithm arranged according to the operation scene image, including a disassembly operation and an assembly operation. The image recognition algorithm includes a machine learning algorithm, a deep learning algorithm, and the like, such as a convolutional neural network, a cyclic neural network, a support vector machine, and the like.

The remote doctor control console and the auxiliary operation device and the patient operation platform and the auxiliary operation device can be identified through the external image acquisition device and the image identification algorithm, and the disassembly and assembly triggering component, such as a ranging sensor and a pressure sensor, can be additionally arranged at the joint of the patient operation platform and the auxiliary operation device to identify the disassembly and assembly operation, and can also confirm the disassembly and assembly operation according to the detection result through the detection result of the connection state of the remote operation robot to the auxiliary operation device.

In addition, the above-mentioned disassembly and assembly operations refer to operations of disassembly and assembly in a physical state, and the interactive operations between the remote doctor console and the auxiliary operation device, and between the patient operation platform and the auxiliary operation device may also include start and stop operations, which refer to operations of starting and stopping in a control state. The start-stop operation may be implemented by a start-stop button/rocker operation, or a start-stop display touch operation. The operation of starting and stopping in the control state may be, for example, an operation of starting and stopping the power output of current, voltage, light, electromagnetic, or the like.

In one embodiment, the first control state comprises a third control state of the remote doctor control platform and a fourth control state of the remote doctor control platform, wherein a master-slave state priority is preset between the third control state and the fourth control state, the center server is in communication connection with one end of the remote doctor control platform and one end of the patient operation platform, the third control state is sent to the patient operation platform through the remote doctor control platform, one control state is selected from the current fourth control state and the received third control state according to the master-slave state priority by the patient operation platform, or the fourth control state is sent to the remote doctor control platform through the patient operation platform, one control state is selected from the current third control state and the received fourth control state according to the master-slave state priority by the remote doctor control platform, and the bandwidth of the communication connection between the remote doctor and the patient operation platform is adjusted through the center server according to the remote control platform or the selected control state of the remote doctor control platform.

In this embodiment, the central server is only in communication connection with the remote doctor console or the patient operating platform, the third control state and the fourth control state are collected to one end of the two, and the collected one end uniformly sends a first bandwidth adjustment request to the central server based on the selected control state to adjust the bandwidth of the network channel.

Specifically, the remote doctor control console further stores a third control state triggered by a history, when the patient operation platform sends the fourth control state to the remote doctor control console, one of the control states is selected to determine whether bandwidth adjustment is needed, or the patient operation platform further stores the fourth control state triggered by the history to represent the current operation stage and operation task of the remote operation robot, and when the remote doctor control console sends the third control state to the patient operation platform, one of the control states is selected to determine whether bandwidth adjustment is needed.

In one embodiment, the master-slave state priority may also be represented by a control state priority, where the connection state with the lower matched bandwidth value is higher, such as the third control state representing the post-operation control state of the tele-surgical robot, the matched bandwidth being 20Mbps, the fourth control state representing the intra-operation control state of the tele-surgical robot, the matched bandwidth being 60Mbps, in the third control state and the fourth control state, which represents that the tele-surgical robot should end the operation, and the priority of the third control state is higher. The foregoing examples of the relative control state priorities also apply to the examples of the master-slave state priorities herein, and thus are not repeated.

Wherein one of the third control state and the fourth control state is selected from the third control state and the fourth control state according to the master-slave state priority from high to low.

In one embodiment, the master-slave state priority may also be represented by a control state priority, where the control state with a lower matched bandwidth value is higher, such as the fourth control state is a post-operation control state, the matched bandwidth is 20Mbps, the third control state is an intra-operation control state, and the matched bandwidth is 60Mbps, where the remote operation robot should end the operation, and the fourth control state has a higher priority. The foregoing examples of the relative control state priorities also apply to the examples of the master-slave state priorities herein, and thus are not repeated.

The central server is in communication connection with the remote doctor control console and the patient operation platform respectively, the third control state is sent to the central server through the remote doctor control console, the fourth control state is sent to the central server through the patient operation platform, one of the third control state and the fourth control state is selected according to the master-slave state priority through the central server, and the bandwidth of communication connection between the remote doctor control console and the patient operation platform is adjusted according to the selected control state.

In this embodiment, the central server may also be simultaneously connected to the remote doctor console and the patient operating platform in communication, and the third control state and the fourth control state may be collected in the central server, and based on the selected control states, the bandwidth of the network channel may be adjusted.

Referring to fig. 11, a flow chart of a bandwidth adjustment method is shown below, and a preferred embodiment of the bandwidth adjustment method is provided, which is specifically as follows:

The tele-doctor console can determine the surgical task 1 based on the interaction operation and further determine the surgical stage 1 based on the interaction operation during the interaction operation for the patient surgical platform, so as to determine the tele-surgical robot as the third control state, and likewise, the patient surgical platform can determine the surgical task 2 and further determine the surgical stage 2 and further determine the tele-surgical robot as the fourth control state during the interaction operation for the tele-doctor console. One of the control states is selected based on the master-slave state priority.

The remote doctor control console is detachably connected with an auxiliary operation device A, a second control state of the remote operation robot can be determined in the interaction operation process of the auxiliary operation device A and the auxiliary operation state B, and the second control state of the remote operation robot can also be determined in the interaction operation process of the auxiliary operation state B and the auxiliary operation state B.

And further selecting a control state according to the control state priority according to the control state selected based on the master-slave state priority and the second control state, wherein the control state is used for initiating a first bandwidth adjustment request to the central server so that the central server initiates a second bandwidth adjustment request to the network operator server, and the network operator server adjusts the bandwidth between the remote doctor control console and the patient operation platform based on the second bandwidth adjustment request.

5. Dynamic adjustment of bandwidth based on auxiliary surgical device

The bandwidth adjustment system comprises at least one auxiliary operation device, a tele-operation robot, a central server, a processor and a control system, wherein the tele-operation robot comprises a first terminal and a second terminal which are in communication connection, one end of the first terminal and one end of the second terminal is a tele-doctor control console, the other end of the first terminal and one end of the second terminal are patient operation platforms, at least one auxiliary operation device is detachably connected to at least one end of the first terminal and one end of the second terminal, the central server is in communication connection with at least one end of the first terminal and one end of the second terminal, the processor is configured to trigger a first connection state between the first terminal and the second terminal in response to interaction operation executed by the first terminal or the second terminal and the auxiliary operation device, and bandwidth of communication connection between the first terminal and the second terminal is adjusted through the central server according to the first connection state.

Specifically, fig. 12 illustrates a bandwidth adjustment flow for a first terminal and a second terminal when only the first terminal is communicatively connected to a center server. The second terminal sends a first control state triggered by the second terminal to the first terminal, the first terminal checks the first control states triggered by the first terminal and the second terminal in real time according to the operation process/a preset period, when the first connection state is kept unchanged, bandwidth adjustment is not needed, and when the first connection state is changed, bandwidth adjustment is requested to the central server based on the changed first connection state.

In this embodiment, the interaction operation includes a start-stop operation and a disassembly operation, and the first connection state between the first terminal or the second terminal and the auxiliary operation device is triggered in response to the start-stop operation or the disassembly operation performed by the first terminal or the second terminal and the auxiliary operation device.

Specifically, the auxiliary operation device triggers the start-stop operation in response to a touch operation on the graphical user interface or a physical operation on the physical hardware. Wherein the operation components such as buttons/rockers, etc., the graphical user interface may be displayed on the touch screen.

The external image acquisition device is in communication connection with a first terminal or a second terminal detachably connected with the auxiliary operation device, acquires an operation scene image through the external image acquisition device, wherein the operation scene image comprises the auxiliary operation device and an image of the first terminal or the second terminal detachably connected with the auxiliary operation device, and identifies the dismounting operation of the first terminal or the second terminal and the auxiliary operation device through the first terminal or the second terminal detachably connected with the auxiliary operation device according to the operation scene image.

Referring to fig. 13, a flowchart of a bandwidth adjustment method is provided, and one embodiment of the bandwidth adjustment method is as follows:

1301. triggering a first connection state between the first terminal or the second terminal and the auxiliary surgical device in response to an interactive operation executed by the first terminal or the second terminal and the auxiliary surgical device;

In this embodiment, at least one of the first terminal and the second terminal stores a historical first connection state, each of the first connection states is preset with a connection state priority, one of the first connection states is selected from a first connection state triggered currently and a historical first connection state according to the connection state priority through at least one of the first terminal and the second terminal, and a bandwidth of communication connection between the first terminal and the second terminal is adjusted through the central server according to the selected first connection state.

Specifically, the central server is only in communication connection with the remote doctor control console or the patient operation platform, the first connection states generated by the remote doctor control console or the patient operation platform are collected to one end of the remote doctor control console or the patient operation platform, one of the collected first connection states is selected by one end based on the connection state priority, a first bandwidth adjustment request is uniformly sent to the central server, and the bandwidth of the network channel is adjusted.

Specifically, the remote doctor control console also stores a first connection state triggered by history to indicate the current surgical stage and surgical task of the remote surgical robot, and when the patient surgical platform transmits the first connection state generated by the patient surgical platform to the remote doctor control console, one of the first connection states is selected to determine whether bandwidth adjustment is required, or the patient surgical platform also stores the first connection state triggered by history to indicate the current surgical stage and surgical task of the remote surgical robot, and when the remote doctor control console transmits the first connection state to the remote doctor control console, one of the first connection states is selected to determine whether bandwidth adjustment is required.

In one embodiment, the connection state priority may also be represented by a control state priority, where the priority of the data type with a lower bandwidth value is higher in the first connection state, for example, the first connection state triggered by the history represents a post-operation stage, the bandwidth of the match is 20Mbps, the first connection state triggered currently represents an intra-operation stage, the bandwidth of the match is 60Mbps, and it is indicated that the tele-operation robot should end the operation at this time, the priority of the first connection state triggered by the history is higher, and the bandwidth of the match is adjusted to 20Mbps. The foregoing examples of the relative control state priorities also apply to the examples of the connection state priorities herein, and thus are not repeated.

And selecting one of the first connection states from the first connection state triggered by the history and the first connection state triggered currently according to the priority of the connection states from high to low.

In this embodiment, the central server may also be simultaneously connected to the remote doctor console and the patient operation platform in a communication manner, where the first connection states generated by the remote doctor console and the patient operation platform are collected in the central server, and the central server selects the first connection state according to the connection state priority, so as to adjust the bandwidth of the network channel.

Further, the central server may also store historical first connection states, and after receiving the first connection states sent by any one of the remote doctor console and the patient operation platform, select one of the first connection states based on the connection state priority, and adjust the bandwidth of the network channel.

1302. And according to the first connection state, regulating the bandwidth of the communication connection between the first terminal and the second terminal through the central server.

In one embodiment, a first control state of the tele-surgical robot is triggered in response to an interactive operation of the first terminal with respect to the second terminal, a second control state of the tele-surgical robot is triggered in response to the first connection state, when the first control state and the second control state are triggered simultaneously, one of the first control state and the second control state is selected according to a preset control state priority through at least one end of the first terminal and the second terminal, and a bandwidth of a communication connection between the first terminal and the second terminal is adjusted through the central server according to the selected control state.

In one embodiment, a second control state of the tele-surgical robot is triggered in response to the first connection state, and a bandwidth of a communication connection between the first terminal and the second terminal is adjusted by the central server in accordance with the second control state.

The method for adjusting the bandwidth based on the auxiliary operation device comprises the following three steps of 1) directly triggering a second control state and adjusting the bandwidth based on the interaction operation of the first terminal or the second terminal and the auxiliary operation device, 2) directly triggering a first connection state and adjusting the bandwidth based on the interaction operation, and 3) further determining the second control state based on the first connection state and adjusting the bandwidth according to the second control state.

In the embodiment, the auxiliary operation device comprises a first auxiliary operation device detachably connected with the remote doctor control console and a second auxiliary operation device detachably connected with the patient operation platform, wherein the first connection state comprises a second connection state triggered based on the interaction operation of the remote doctor control console and the first auxiliary operation device and a third connection state triggered based on the interaction operation of the patient operation platform and the second auxiliary operation device, a master-slave state priority is preset between the second connection state and the third connection state, one connection state is selected from the second connection state and the third connection state according to the master-slave state priority through the remote doctor control console or the patient operation platform, and the bandwidth of communication connection between the remote doctor control console and the patient operation platform is adjusted through the central server according to the selected connection state.

In this embodiment, the central server is only in communication connection with the remote doctor console or the patient operation platform, the second connection state and the third connection state are collected to one end of the two, and the collected one end uniformly sends a first bandwidth adjustment request to the central server based on the selected connection state to adjust the bandwidth of the network channel.

Specifically, the remote doctor control console also stores a second connection state triggered by history, and when the patient operation platform sends the third connection state to the remote doctor control console, one of the connection states is selected for determining whether bandwidth adjustment is needed, or the patient operation platform also stores the third connection state triggered by history, which indicates the current operation stage and operation task of the remote operation robot, and when the remote doctor control console sends the second connection state to the remote doctor control console, one of the connection states is selected for determining whether bandwidth adjustment is needed.

In one embodiment, the master-slave state priority may also be represented by a control state priority, where the connection state with the lower bandwidth value matched is higher, such as the third connection state representing the post-operation control state of the tele-surgical robot, the bandwidth matched is 20Mbps, the second connection state representing the intra-operation control state of the tele-surgical robot, the bandwidth matched is 60Mbps, and the priority of the third connection state is higher. The foregoing examples of the relative control state priorities also apply to the examples of the master-slave state priorities herein, and thus are not repeated.

In addition, the central server can be simultaneously and respectively connected with the remote doctor control console and the patient operation platform in a communication mode, the second connection state and the third connection state are collected in the central server, and the bandwidth of the network channel is adjusted based on the connection state selected by the priority of the master state and the slave state.

In this embodiment, the central server is in communication connection with a preset network operator server, and the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server according to the first connection state, wherein the bandwidth adjustment request comprises matching bandwidth parameters corresponding to the first connection state through at least one end of the first terminal and the second terminal, sending a first bandwidth adjustment request to the central server based on the bandwidth parameters, and initiating a second bandwidth adjustment request to the network operator server based on the first bandwidth adjustment request through the central server so as to adjust the bandwidth of the communication connection between the first terminal and the second terminal through the network operator server.

In this embodiment, the bandwidth parameter may be matched with the first connection state in addition to the first control state, so as to reduce the association steps between the first control state and the first connection state. And the first connection state is matched to obtain the bandwidth parameter of the corresponding second control state.

Further, at least one end of the first terminal and the second terminal stores mapping information, the bandwidth parameter comprises a plurality of bandwidth levels according to a bandwidth high-low sequence, the mapping information comprises a plurality of mapping relations between the first connection states and the bandwidth levels, and a first bandwidth adjustment request is sent to the central server based on the bandwidth parameter, wherein the first bandwidth adjustment request is sent to the central server based on the bandwidth parameter, and comprises that a target bandwidth level corresponding to the first connection state is matched in the stored mapping information through at least one end of the first terminal and the second terminal, and the first bandwidth adjustment request is initiated to the central server based on the target bandwidth level.

In the embodiment, the bandwidth levels of different first connection states are set, the preset bandwidth sizes of different operation stages and operation tasks are determined, the bandwidth preset is higher, the bandwidth preset is smaller, the bandwidth preset is lower, for example, 4 bandwidth levels from low to high are set, namely 20Mbps, 30Mbps, 40Mbps and 60Mbps, namely { 20Mbps before operation, 60Mbps in operation, 20Mbps after operation, 60Mbps of operation tasks, 40Mbps of debugging tasks and 30Mbps of teaching tasks }.

In addition, when the remote doctor control console and the patient operation platform both store mapping information, the remote doctor control console and the patient operation platform are respectively matched with target bandwidth levels of the first connection state of the remote doctor control console and the patient operation platform, and respectively initiate first bandwidth adjustment requests to the central server based on the respective target bandwidth levels.

When only the remote doctor control console stores mapping information, the patient operation platform sends the first connection state of the remote doctor control console to the remote doctor control console, the remote doctor control console is matched with the target bandwidth level of the first connection state of the patient operation platform and the remote doctor control console, and at the moment, the remote doctor control console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

When only the patient operation platform stores mapping information, the remote doctor control platform sends the first connection state of the patient operation platform to the patient operation platform, the patient operation platform is matched with the target bandwidth level of the first connection state of the remote doctor control platform and the patient operation platform in a unified mode, and at the moment, the patient operation platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

In addition, the mapping information can be stored in the central server, and the central server uniformly matches the target bandwidth level corresponding to the first connection state of the remote doctor console and the patient operation platform according to the stored mapping information, and directly initiates a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

In this embodiment, the auxiliary operation device includes a first auxiliary operation device and a second auxiliary operation device with different function types, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted by the central server according to a preset first amplitude according to a first connection state triggered by the first auxiliary operation device, and the bandwidth of the communication connection between the first terminal and the second terminal is adjusted by the central server according to a preset second amplitude according to a first connection state triggered by the second auxiliary operation device.

In the embodiment, the first auxiliary operation device has a function type that the auxiliary operation device needs to provide the function of the operation item in a whole uninterrupted manner, and the second auxiliary operation device has a function type that the auxiliary operation device does not need to provide the function of the operation item in a whole uninterrupted manner. Based on the foregoing distinction of the two function types, specific examples are as follows:

A. The auxiliary operation device detachably connected with the remote doctor control console comprises endoscope control head equipment, brain electricity monitoring head equipment, eye movement tracking equipment, a display large screen and the like. If the electroencephalogram monitoring head-mounted equipment and the eye movement tracking equipment are used for controlling the suction head of the liquid suction system to move, the liquid suction system is required to be used in the whole course in the current operation project, the endoscope controls the head-mounted equipment, the electroencephalogram monitoring head-mounted equipment and the eye movement tracking equipment to belong to a first auxiliary operation device, and the display large screen belongs to a second auxiliary operation device. If the liquid suction system is not needed to be used in the whole process in the current operation project, the endoscope control head-mounted equipment belongs to a first auxiliary operation device, and the brain electricity monitoring head-mounted equipment, the eye movement tracking equipment and the display large screen belong to a second auxiliary operation device.

B. The auxiliary operation device for detachable connection with the patient operation platform comprises an operation instrument, a catheter instrument, an irrigation system, an operation table, a sterile cover, a poking card, an energy platform, a monitor, a liquid suction system, a pneumoperitoneum machine and the like. The first auxiliary operation device comprises an operation table, a sterile cover, a poking card and a monitor, and the second auxiliary operation device comprises an operation instrument, a catheter instrument, an irrigation system, an energy platform, a liquid suction system and a pneumoperitoneum machine.

The method comprises the steps of monitoring duration of a first connection state triggered by a first auxiliary operation device through at least one of the first terminal and the second terminal, and adjusting the bandwidth of communication connection between the first terminal and the second terminal according to a preset first amplitude through the central server when the duration exceeds a preset first duration.

For the connection relation between a first auxiliary operation device such as an operation table, a sterile cover, a stamping card and a monitor and a patient operation platform, the first connection state is determined to be the unconnected state (belonging to the preoperative control state) when the first auxiliary operation device is unconnected, the bandwidth is adjusted to be the connected state (belonging to the intraoperative control state) when the first auxiliary operation device is connected, the bandwidth is adjusted to be 60Mbps when the first auxiliary operation device is connected, the first connection state is determined to be the detached state (belonging to the postoperative control state) when the first auxiliary operation device is detached, and the bandwidth is adjusted to be 10Mbps when the first auxiliary operation device is detached.

Specifically, according to different first connection states of the first auxiliary operation device, different first time periods are respectively set to be 0, 17ms, 60ms and the like in an unconnected state and a connection state, and set to be 1min, 2min, 5min, 10min and the like in a detached state. The former has real-time switching bandwidth to meet the operation requirement, and the latter has obvious time interval to prevent the operation interruption and the operation from being continued.

For a second auxiliary surgical device, such as an electroencephalogram monitoring head-mounted device, an eye movement tracking device, a display large screen, a surgical instrument, a catheter instrument, an irrigation system, an energy platform, a liquid suction system, and a pneumoperitoneum machine. According to different first connection states of the second auxiliary operation device, different second time periods are respectively set to be shorter time periods of 0, 17ms, 60ms and the like in the unconnected state and the connection state, and in the disassembly state, the required operation time period (namely static time period) of each operation item or the residual operation time period (namely dynamic time period) of the current operation process can be set, for example, the required operation time period of the operation item is 180min, the second time period can be directly set to be 180min, and the second time period can be set to be 130min after 50min of the current operation process. I.e. the former has a real-time switching bandwidth to meet the surgical demands, and the latter has a significant time interval, since the second auxiliary surgical device, although detached, cannot indicate the end of the surgery, is required to wait for the required surgical duration before adjusting the bandwidth. Wherein, the preset first time length is also set to be smaller than the rest operation time length.

The method comprises the steps of monitoring duration of a first connection state triggered by a second auxiliary operation device through at least one of a first terminal and a second terminal, and adjusting bandwidth of communication connection between the first terminal and the second terminal according to a preset second amplitude through the central server when the duration exceeds a preset second duration, wherein the first amplitude is larger than the second amplitude, and the preset first duration is smaller than the preset second duration.

For a first auxiliary surgical device, representing a switching of connection states for different surgical phases, transmission involving a larger amount of data increases/decreases, so a larger first amplitude adjustment bandwidth is required. Such as connection state switching representing preoperative (10 Mbps), intra-operative (60 Mbps) and post-operative (10 Mbps), or connection state switching representing power-on self-test (10 Mbps), remote pairing/remote connection (20 Mbps), remote interactive communication/remote surgical operation (60 Mbps), remote pairing release/remote pairing switching (20 Mbps), and power-off self-test (10 Mbps).

For a second auxiliary surgical device, representing a connection state switch for the same surgical phase, transmission involving a smaller amount of data increases/decreases, so a smaller second amplitude adjustment bandwidth is required. For example, only partial data transmission related to brain monitor head-mounted equipment, eye movement tracking equipment, display large screen, surgical instruments, catheter instruments, flushing systems, energy platforms, liquid suction systems and pneumoperitoneum machines is reduced in operation, the occupied bandwidth of the partial data transmission is smaller, and video data and voice data (such as occupying 90% of bandwidth) with larger occupied bandwidth in operation are still continuously transmitted.

6. Dynamic adjustment of bandwidth based on data type transmission

The bandwidth adjustment system comprises a first remote host and a second remote host, wherein a network channel is established between the first remote host and the second remote host, a tele-surgical robot comprises a first terminal connected with the first remote host and a second terminal connected with the second remote host, one end of the first terminal and one end of the second terminal are remote doctor consoles, the other end of the first terminal and the second terminal are patient surgical platforms, a central server is connected to at least one of the first remote host and the second remote host, a processor is configured to respond to first interaction operation performed by the first terminal on the second terminal, generate first interaction data, transmit the first interaction data to the second remote host through the network channel by the first remote host, enable the second terminal to perform motion control based on the first interaction data, identify a first interaction data type transmitted in the network channel by at least one of the first remote host and the second remote host, and adjust the bandwidth of the network channel by the central server according to the first interaction data type.

Specifically, fig. 14 illustrates a bandwidth adjustment flow for the first terminal and the second terminal when only the first terminal is communicatively connected to the center server. The first terminal checks/transmits the first data type of the first interactive data transmitted in the network channel according to the operation process in real time, as shown in fig. 14, the first interactive data is generated by itself and transmitted to the second remote host through the first remote host, and the first data type is transmitted to the first remote host for forwarding, so that the first data type can be identified through the first remote host. When the first data type is changed, the bandwidth is required to be adjusted based on the changed first data type.

Referring to fig. 15, a flowchart of a bandwidth adjustment method is provided, and one embodiment of the bandwidth adjustment method is as follows:

1501. Generating first interaction data in response to a first interaction operation executed by the first terminal for the second terminal, and transmitting the first interaction data to the second remote host through the first remote host via the network channel so as to enable the second terminal to perform motion control based on the first interaction data;

In this embodiment, the first interaction data at least includes at least one of endoscope video data, operating room cradle head video data, operating room rifle bolt video data, audio data, master-slave control signaling data, pairing data, energy control data, and the like.

The remote doctor control console and the patient operation platform each perform first interactive operations of starting an endoscope video function, an operating room cradle head video function, an operating room gun camera video function, a voice call function and a pairing function, transmitting corresponding video data, audio data and pairing data based on the started functions, and transmitting control signaling and energy control data when the pedal, an operating component such as a clamping jaw/a finger ring and a handle perform the first interactive operations.

When at least one end of the remote doctor control console and the patient operation platform execute first interactive operation, the target end performs corresponding motion control. For example, based on the fact that the two ends simultaneously open an endoscope video function, an operating room cradle head video function and an operating room gun camera video function, a remote doctor control console displays the endoscope video, the operating room cradle head video and the operating room gun camera video; the remote doctor control console selects a control mode through a pedal, and performs operation through an operation part, so that a mechanical arm of a patient operation platform and connected surgical instruments and energy instruments move along with the mechanical arm.

1502. Identifying, by at least one of the first remote host and the second remote host, a first data type of first interaction data transmitted in the network channel;

In this embodiment, the first interaction data is transmitted between the remote doctor console and the patient operating platform through the network channel established between the first remote host and the second remote host, so that the first data type of the first interaction data transmitted in the network channel can be directly identified at the first remote host and the second remote host.

The amount of data for the first interactive data transmission is different for each first data type, and therefore the amount of bandwidth required for transmission in the network channel is also different. For example, when transmitting in the network channel, the endoscope video needs to occupy 30Mbps of bandwidth, the operating room cradle head video needs to occupy 10Mbps of bandwidth, the operating room gun camera video needs to occupy 10Mbps of bandwidth, the audio data needs to occupy 20Mbps of bandwidth, the master-slave control signaling data needs to occupy 10Mbps of bandwidth, the pairing data needs to occupy 5Mbps of bandwidth, the energy control data needs to occupy 5Mbps of bandwidth, etc. The actual required bandwidth value may be set based on demand and is not particularly limited herein.

In this embodiment, at least one end of the first remote host and the second remote host is used to obtain a historical interaction data type transmitted in the network channel, compare the first data type with the historical interaction data type, determine a switching state of the first data type transmitted in the network channel, and adjust a bandwidth of the network channel through the central server according to the switching state. Wherein, the historical interaction data type refers to the latest interaction data type transmitted last time.

The switching state of the first data type transmitted in the network channel is that the paired data type is switched to the endoscope video data type. Based on the switching state, the center server adjusts 5 Mbps of the bandwidth required for the network channel to transmit the paired data to 30Mbps of the bandwidth required for transmitting the endoscopic video data. The first data type currently transmitted is the paired data type, and the switching state is that the paired data type is kept unchanged. The central server does not need to regulate the bandwidth of the network channel. Wherein a plurality of first data types can be transmitted in the network channel, and the bandwidth of the network channel is adjusted according to the sum of the required bandwidths of each first data type.

In one embodiment, the switching states include a first switching state and a second switching state, wherein the first switching state includes a state of switching a data type at a different surgical stage or surgical task, the second switching state includes a state of switching a data type at the same surgical stage or surgical task, the duration of the first switching state is monitored, the bandwidth of the network channel is adjusted according to a preset first amplitude when the duration exceeds a preset first duration, or the duration of the second switching state is monitored, the bandwidth of the network channel is adjusted according to a preset second amplitude when the duration exceeds a preset second duration, and the first amplitude is greater than the second amplitude, and the preset first duration is less than the preset second duration.

In this embodiment, the first switching state and the second switching state respectively preset the first data type before switching and the first data type after switching, and the simple examples are as follows:

1) In the first switching state, the first data type before switching comprises a pairing data type, and the first data type after switching comprises at least one of an endoscope video data type, an operating room cradle head video data type, an operating room gun camera video data type, an audio data type and a master-slave control signaling data type.

2) In the first switching state, the first data type before switching comprises at least one of an endoscope video data type, an operating room cradle head video data type, an operating room gun camera video data type, an audio data type and a master-slave control signaling data type, and the first data type after switching comprises a pairing data type.

3) In the second switching state, the first data type before switching comprises at least one of an endoscope video data type, an operating room cradle head video data type, an operating room gun camera video data type, an audio data type and a master-slave control signaling data type, and the first data type after switching comprises at least one of an endoscope video data type, an operating room cradle head video data type, an operating room gun camera video data type, an audio data type and a master-slave control signaling data type.

In this embodiment, after the first data type is switched, it indicates that the current tele-surgical robot enters the next surgical stage or enters a different surgical task, that is, the first switching state indicates that the surgical stage is switched or the surgical task is switched. This involves a transmission increase/decrease of a larger data volume while switching to the next surgical phase in time, so a shorter first duration, and a larger first amplitude adjustment bandwidth is required.

By way of example, the first interaction data may be, for example, pre-operative pairing data (10 Mbps), intra-operative endoscopic video data, operating room cradle head video data, operating room gun camera video data, audio data, master-slave control signaling data, energy control data (60 Mbps), post-operative pairing data (10 Mbps), and thus, a first switching state of a first data type pre-operative, intra-operative and post-operative requires a first shorter duration (e.g., real-time switching, 17ms, 60ms, etc.), and a first larger amplitude adjustment bandwidth (e.g., + -50 Mbps).

In this embodiment, after the first data type is switched, it indicates that the current tele-surgical robot is still in the same surgical stage or surgical task, that is, the second switching state indicates the state switching under the same surgical stage or surgical task. This involves an increase/decrease in the transmission of a smaller amount of data while preventing the situation where the surgical interruption needs to continue, has not yet switched to the next surgical stage, so a longer second duration, and a smaller second amplitude adjustment bandwidth, is required.

By way of example, the first interactive data may be, for example, endoscopic video data (30 Mbps) during the surgical procedure, operating room cradle head video data (10 Mbps), operating room gun camera video data (10 Mbps), audio data (20 Mbps), master-slave control signaling data (10 Mbps), and energy control data (5 Mbps), and during each transmission, one or more first interactive data may be involved, and part of the first interactive data may be temporarily stopped from transmission, and may not be indicative of the end of the surgical procedure, and may enter the post-operative phase. Thus, the second switching state of the multiple first data types in operation requires a longer first duration (e.g., 3 hours of duration required for the procedure item) and a smaller first amplitude adjustment bandwidth (e.g., 5Mbps, 10Mbps, 20 Mbps).

Providing a specific example, when the first interaction data is transmitted over time T (T1, T2, &..the term tn, T1-T2 is a standard duration), determining a first switching state M1 and a second switching state M2, and maintaining a third switching state M0 of the original data type, where the first duration includes zero standard durations and the second duration includes i standard durations, then:

t1 is { the first interaction data is pairing data, the switching state is M0, and the bandwidth is 10Mbps };

t2 is { first interaction data, namely endoscope video data, operating room cradle head video data, operating room gun camera video data and audio data, wherein the switching state is M1, and the bandwidth is 70Mbps };

t3 is { first interaction data, namely endoscope video data, operating room cradle head video data and operating room gun camera video data, wherein the switching state is M2, and the bandwidth is 70Mbps };

t4 is { first interaction data, namely endoscope video data, operating room cradle head video data and operating room gun camera video data, wherein the switching state is M2, and the bandwidth is 70Mbps };

......

ti+3 { first interaction data: endoscope video data, switching state: M2, bandwidth: 50Mbps };

Ti+4 { first interaction data, endoscope video data, pairing data, switching state M1, bandwidth 10Mbps }.

Further, the operation stage at least comprises any one of a start-up self-check, remote pairing, remote connection, remote interactive communication, remote operation, remote pairing release and shutdown self-check, and the operation task comprises any one of an operation task, a teaching task and a debugging task.

In this embodiment, whether the first data type includes a preset target data type is determined, configuration information of first interaction data of the preset target data type is detected through at least one of the first remote host and the second remote host, and bandwidth of the network channel is adjusted through the central server according to the configuration information.

Specifically, the target data type at least comprises endoscope video data, operating room cradle head video data, operating room gun camera video data and the like, and the configuration information of the video data at least comprises image resolution, video coding format and video transmission code rate. Examples of the correspondence between configuration information and bandwidth (the corresponding bandwidth contains occupancy of other data types) are { image resolution: 4k, video coding format: H265, video transmission rate: 6000, bandwidth: 100Mbps }, { image resolution: 4k, video coding format: H265, video transmission rate: 3000, bandwidth: 70Mbps } { image resolution: 2k, video coding format: H265, video transmission rate: 2500, bandwidth: 50Mbps }.

Further, the correspondence between configuration information and bandwidth may also be determined based on different surgical task types. The corresponding relation between the configuration information and the bandwidths can be determined based on the operation task, and the teaching task sets lower corresponding bandwidths based on the same configuration information, namely, the image resolution is 4k, the video coding format is H265, the video transmission rate is 6000, the bandwidths are 20Mbps, the image resolution is 4k, the video coding format is H265, the video transmission rate is 3000, the bandwidths are 10Mbps, the image resolution is 2k, the video coding format is H265, the video transmission rate is 2500, and the bandwidths are 10 Mbps. The debugging task may be set to be the same correspondence between configuration information and bandwidth as the surgical operation task or the same correspondence between configuration information and bandwidth as the teaching task.

In the embodiment, the first interactive data comprises third interactive data transmitted from the remote doctor control console to the patient operation platform and fourth interactive data transmitted from the patient operation platform to the doctor control console, wherein a master-slave data type priority is preset between a third data type of the third interactive data and a fourth data type of the fourth interactive data, the central server is respectively in communication connection with at least one end of the remote doctor control console and the patient operation platform, one data type is selected from the third data type and the fourth data type according to the master-slave data type priority, and the bandwidth of the network channel is adjusted through the central server according to the selected third data type or fourth data type.

In this embodiment, when the central server is only in communication connection with the remote doctor console or the patient operation platform, the third interactive data and the fourth interactive data are collected at one end of the two, and the collected one end uniformly sends the first bandwidth adjustment request to the central server through the remote host based on the selected data type, so as to adjust the bandwidth of the network channel.

The remote doctor control console also stores historical third interaction data which indicates the current surgical stage and surgical task of the remote surgical robot, selects one data type for determining whether bandwidth adjustment is needed when the patient surgical platform sends fourth interaction data to the remote doctor control console, or stores historical fourth interaction data which indicates the current surgical stage and surgical task of the remote surgical robot, and selects one data type for determining whether bandwidth adjustment is needed when the remote doctor control console sends third interaction data to the patient surgical platform.

In one embodiment, the priority of the master-slave data types may also be represented by a control state priority, where the priority of the data types with lower matched bandwidth values in the third interactive data and the fourth interactive data is higher, for example, the fourth interactive data represents a post-operation stage, the matched bandwidth is 20Mbps, the third interactive data represents an intra-operation stage, the matched bandwidth is 60Mbps, and the remote operation robot should end the operation at this time, the priority of the fourth data type is higher, and the bandwidth is adjusted to 20Mbps. The foregoing examples of the relative control state priorities also apply to the examples of master-slave data type priorities herein, and therefore are not repeated.

Wherein one of the third data type and the fourth data type is selected from the third data type and the fourth data type according to the priority of the master data type and the slave data type from high to low.

In this embodiment, the central server may also be simultaneously connected to the remote doctor console and the patient operation platform in communication, and the third interactive data and the fourth interactive data may be collected in the central server, and based on the selected data type, the bandwidth of the network channel may be adjusted.

1503. And adjusting the bandwidth of the network channel through the central server according to the first data type.

In this embodiment, the central server is communicatively connected to a preset network operator server, matches, through at least one end of the first remote host and the second remote host, a bandwidth parameter corresponding to the first data type, and sends a first bandwidth adjustment request to the central server based on the bandwidth parameter, and initiates, through the central server, a second bandwidth adjustment request to the network operator server based on the first bandwidth adjustment request, so as to adjust a bandwidth of the network channel through the network operator server.

In this embodiment, the bandwidth parameter may be matched with the first control state and the first connection state, and may also be matched with the first data type, and the corresponding bandwidth parameter may be obtained based on the first connection state matching.

In one embodiment, the first interactive data includes a plurality of first data types, the plurality of first data types are preset with data type priorities, at least one first data type is selected from the plurality of first data types according to the data type priorities through at least one end of the first remote host and the second remote host, and corresponding bandwidth parameters are matched according to the selected first data type.

In this embodiment, the preset data type priority may be represented as a control state priority, where the matched bandwidth value is lower and the priority is higher among the plurality of first data types representing different surgical phases. For example, the paired data is transmitted in the postoperative stage, the bandwidth matched with the paired data type is 20Mbps, the endoscope video data is transmitted in the intraoperative stage, the bandwidth matched with the endoscope video data type is 60Mbps, the fact that the tele-operation robot is in the unpaired relation at this time is indicated, the operation is finished, the priority of the paired data type is higher, and the bandwidth is adjusted to 20Mbps. The foregoing examples of the relative control state priorities also apply to the examples of the data type priorities herein, and thus are not repeated.

Further, the priority of the data types may also be expressed as the importance of each first data type, the higher the importance, the higher the priority. The method comprises the steps of sequentially including an endoscope video data type, a master-slave control signaling data type, an energy control data type, an audio data type, an operating room holder video data type, an operating room gun camera video data type and a pairing data type from high to low, calculating all data types corresponding to the data type according to the data type with the highest priority, and matching bandwidth parameters of all data types.

In one embodiment, at least one end of the first terminal and the second terminal stores mapping information, the bandwidth parameter comprises a plurality of bandwidth levels according to a bandwidth high-low sequence, the mapping information comprises a plurality of mapping relations between the first data types and the bandwidth levels, a target bandwidth level corresponding to the first data type is matched in the mapping information stored in the corresponding first terminal and second terminal through at least one end of the first remote host and the second remote host, and a first bandwidth adjustment request is initiated to the central server based on the target bandwidth level.

In the embodiment, the bandwidth levels of different first data types are set, the preset bandwidth sizes of different operation stages and operation tasks are determined, the bandwidth preset is higher, the bandwidth preset is smaller, the bandwidth preset is lower, for example, 4 bandwidth levels from low to high are set, namely 20Mbps, 30Mbps, 40Mbps and 60Mbps, namely { 20Mbps before operation, 60Mbps in operation, 20Mbps after operation, 60Mbps of operation tasks, 40Mbps of debugging tasks and 30Mbps of teaching tasks }.

In addition, when the remote doctor control console and the patient operation platform both store mapping information, the remote doctor control console and the patient operation platform are respectively matched with target bandwidth levels of the first data type output by the remote doctor control console and the patient operation platform, and respectively initiate first bandwidth adjustment requests to the central server based on the respective target bandwidth levels.

When only the remote doctor control console stores mapping information, the patient operation platform sends the first data type to the remote doctor control console, the remote doctor control console is matched with the target bandwidth level of the first data type output by the patient operation platform and the remote doctor control console, and at the moment, the remote doctor control console initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

When only the patient operation platform stores mapping information, the remote doctor control console sends the first data type output by the remote doctor control console to the patient operation platform, the patient operation platform is matched with the target bandwidth level of the first data type output by the remote doctor control console, and at the moment, the patient operation platform initiates a first bandwidth adjustment request to the central server based on the target bandwidth level.

In addition, the mapping information can be stored in the central server, and the central server uniformly matches the target bandwidth level corresponding to the first data type output by the remote doctor console and the patient operation platform according to the stored mapping information, and directly initiates a second bandwidth adjustment request to the network operator server based on the target bandwidth level.

In this embodiment, the method further includes at least one auxiliary operation device, at least one of the first terminal and the second terminal is detachably connected to the at least one auxiliary operation device, second interactive data are generated in response to second interactive operation performed by the first terminal or the second terminal and the auxiliary operation device and transmitted between the first remote host and the second remote host through the network channel, the second data type of the second interactive data transmitted in the network channel is identified through at least one of the first remote host and the second remote host, one of the first data type and the second data type is selected according to a preset relative type priority between the first data type and the second data type, and the bandwidth of the network channel is adjusted through the central server according to the selected first data type or the second data type.

In one embodiment, the preset relative type priority may also be represented as a control state priority, where the matched bandwidth value is lower and the priority is higher in the first data type and the second data type. The preset relative type priority may also be represented as a control state priority, which indicates the importance of the first interactive data relative to the second interactive data, for determining whether the bandwidth needs to be adjusted or the size of the adjusted bandwidth to meet the required bandwidth of the surgical procedure. The foregoing examples of relative control state priorities also apply to the examples of relative type priorities herein, and are not repeated. Wherein one of the first data type and the second data type is selected from the first data type and the second data type according to the relative type priority from high to low.

7. Bandwidth allocation for multiple groups of tele-surgical robots

The bandwidth allocation system comprises a plurality of groups of tele-operation robots, a central server and a processor, wherein each group of tele-operation robots comprises a first terminal and a second terminal which are in communication connection, each group of the first terminal and the second terminal are in communication connection in the same network channel, one end of each group of the first terminal and the second terminal is a tele-doctor control console, the other end of each group of the first terminal and the second terminal is a patient operation platform, the central server is in communication connection with at least one end of each group of the first terminal and the second terminal of each group of the tele-operation robots, the processor is configured to acquire interaction information of each group of the first terminal and the second terminal, determine operation scenes of each group of the tele-operation robots according to the interaction information, allocate bandwidth to each group of the tele-operation robots according to the current bandwidth in each operation scene and the network channel, and adjust the bandwidth of each group of the tele-operation robots.

Exemplary, as shown in fig. 16, includes two groups of tele-surgical robots, the first group of tele-surgical robots including a first terminal TX1 and a second terminal RX1, and the second group of tele-surgical robots including a first terminal TX2 and a second terminal RX2, and exemplary bandwidth allocation for the two groups of tele-surgical robots is performed when only the first terminal TX1 and the first terminal TX2 are communicatively connected with a central server, respectively.

The two groups of remote operation robots respectively collect interaction information to a first terminal TX1 and a first terminal TX2, for example, the second terminal RX1 and the second terminal RX2 respectively send the self-generated interaction information to the first terminal TX1 and the first terminal TX2, the first terminal TX1 and the first terminal TX2 check operation scenes according to operation processes in real time/according to preset periods, the operation scenes are checked based on the collected interaction information, when the operation scenes are kept unchanged, bandwidth adjustment is not required, and when the operation scenes are changed, bandwidth adjustment is required to be carried out to a central server based on the changed operation scenes.

After receiving the request, the central server allocates bandwidth to the two groups of teleoperation robots according to the operation scene of bandwidth adjustment request, initiates a bandwidth adjustment request to the network operator server based on the allocated bandwidth, firstly queries node bandwidths of the first terminal TX1 and the first terminal TX2 to the network operator server and returns the node bandwidths according to the confirmation query information, the central server sends identity information of the central server or the identity information of the first terminal TX1 and the first terminal TX2 to the operator server for authentication, the safety of bandwidth adjustment is ensured, the node bandwidths of the first terminal TX1 and the first terminal TX2 can be adjusted to the requested allocation bandwidth after the authentication is passed, and the central server forwards the adjusted bandwidth to the first terminal TX1 and the first terminal TX2 to prompt the first terminal TX1 and the first terminal TX 2.

Referring to fig. 17, a flowchart of a bandwidth allocation method is provided, and one embodiment of the bandwidth allocation method is as follows:

1701. Acquiring interaction information of the first terminal and the second terminal through each group of remote operation robots, and determining operation scenes of each group of remote operation robots according to the interaction information;

In this embodiment, the interaction information includes at least one of a first control state of the tele-surgical robot and interaction data transmitted in the network channel when the first terminal performs interaction operation on the second terminal, and at least one auxiliary surgical device, and the interaction information includes at least one auxiliary surgical device, and the interaction information may further include a second control state of the tele-surgical robot when the first terminal and/or the second terminal performs interaction operation with the auxiliary surgical device that is detached from the connection, wherein at least one end of the first terminal and at least one end of the second terminal are detachably connected with at least one auxiliary surgical device. And determining the surgical scene of each group of remote surgical robots according to at least one of the first control state, the second control state and the interaction data.

In the embodiment, the operation scene comprises at least one of a scene under different operation stages, a scene under different operation tasks and a scene under different operation types, wherein the operation stages at least comprise any one of a startup self-check, remote pairing, remote connection, remote interaction communication, remote operation, remote pairing release and shutdown self-check, the operation tasks at least comprise any one of an operation task, a teaching task and a debugging task, and the operation types comprise any one of different emergency operation types.

In one embodiment, each group of the tele-surgical robots further comprises a first voice acquisition device corresponding to the first terminal and a second voice acquisition device corresponding to the second terminal, and/or further comprises an in-vivo image acquisition device corresponding to the patient surgical platform. The remote doctor control console and the medical staff of the patient operation platform can communicate through the first voice acquisition device and the second voice acquisition device.

The in-vivo image acquisition device is used for acquiring an anatomical scene image in the surgical process, the voice communication information and the anatomical scene image are transmitted through the network channel, the interaction information comprises the voice communication information and/or the anatomical scene image, a scene of a corresponding tele-surgical robot in any surgical stage, a scene of any surgical task and/or a scene of any surgical type is determined according to the voice communication information, and/or a scene of the corresponding tele-surgical robot in any surgical task and/or a scene of any surgical type is determined according to the anatomical scene image.

Specifically, the in-vivo image acquisition device is arranged at one end of the patient operation platform, can be arranged at the tail end of the operation instrument, enters the patient body along with the operation instrument, and shoots an anatomical scene image in the patient body. The patient operation platform is provided with an image recognition algorithm, and the arranged image recognition algorithm is applied to recognize the scene under any operation task and/or the scene under any operation type according to the anatomical scene image. The image recognition algorithm includes a machine learning algorithm, a deep learning algorithm, and the like, such as a convolutional neural network, a cyclic neural network, a support vector machine, and the like.

Specifically, at least one end of the patient operation platform and at least one end of the remote doctor control console are provided with a voice recognition algorithm, feature keywords are extracted according to voice communication information in the communication process of medical staff at the two ends, and scenes under any operation stage, scenes under any operation task and/or scenes under any operation type are recognized according to the feature keywords.

By way of example, related feature keywords such as "cut", "suture", "imbibition" and the like, a surgical scene currently in an intraoperative phase may be determined. Other feature keywords may be collected based on a priori knowledge, creating a feature keyword library for use in identifying surgical scenarios. The speech recognition algorithm may specifically be a dynamic time plan (DYNAMIC TIME WARPING, DTW) based, a parametric model based hidden Markov model (Hidden Markov Model, HMM), a convolutional neural network, a deep learning neural network, or the like.

In addition, the different operation types are classified by the emergency degree of the operation, and are classified based on at least one of preoperative evaluation information such as patient age, operation risk degree, operation duration, operation site, etc., intraoperative operation information such as patient bleeding condition, whether emergency accident occurs, operation progress, etc.

1702. And allocating bandwidth to each group of remote operation robots according to each operation scene and the current bandwidth in the network channel, wherein the bandwidth is used for adjusting the bandwidth of each group of remote operation robots by the central server.

In the embodiment, a preset required bandwidth of each operation scene is obtained, a total required bandwidth of each preset required bandwidth is determined, the total required bandwidth is compared with the current bandwidth in the network channel, if the total required bandwidth is higher than the current bandwidth in the network channel, at least one of the preset required bandwidth of at least one group of tele-operation robots and the current bandwidth in the network channel is adjusted, so that the adjusted total required bandwidth is not higher than the current bandwidth in the network channel, and the bandwidth allocated to each group of tele-operation robots is determined based on the adjusted preset required bandwidth and/or the current bandwidth.

The operation scene of the first group of remote operation robots is an operation scene in a preoperative stage, the preset required bandwidth is 20Mbps, the operation scene of the second group of remote operation robots is an operation scene in an intraoperative stage, the preset required bandwidth is 80Mbps, the operation scene of the third group of remote operation robots is an operation scene in a postoperative stage, the preset required bandwidth is 20Mbps, and the total required bandwidth is 120Mbps. The current bandwidth in the network channel is 100Mbps, in one scheme the total required bandwidth is regulated to be no higher than 100Mbps, in another scheme the current bandwidth is regulated to be higher than 120Mbps.

When the total required bandwidth is 120Mbps, if the total required bandwidth is adjusted to be not higher than 100Mbps, the preset required bandwidth of the first group of tele-surgical robots (lower than 20 Mbps), the preset required bandwidth of the second group of tele-surgical robots (lower than 80 Mbps), and/or the preset required bandwidth of the third group of tele-surgical robots (lower than 20 Mbps) need to be reduced.

In one embodiment, a scene priority is preset for a plurality of surgical scenes, a bandwidth threshold is preset for the network channel, if the total required bandwidth is higher than the current bandwidth in the network channel, the transmission frequency of at least one group of remote surgical robots is reduced in sequence according to the scene priority, the reduced preset required bandwidth corresponding to at least one surgical scene is determined according to the reduced transmission frequency, the reduced total required bandwidth is determined based on the reduced preset required bandwidth, and if the reduced total required bandwidth is higher than the current bandwidth in the network channel and not higher than the bandwidth threshold, the current bandwidth in the network channel is increased to the reduced total required bandwidth.

In this embodiment, the bandwidth threshold set by the network channel is, for example, 100Mbps, 150Mbps, 200Mbps, 300Mbps, 500Mbps, etc. A bandwidth threshold is set based on the occupied bandwidth of the desired compatible sets of tele-surgical robots. The present embodiment describes preferentially adjusting the transmission frequency of the tele-surgical robot to adjust the total required bandwidth and then to adjust the current bandwidth of the network channel when the total required bandwidth is higher than the current bandwidth.

In the embodiment, for different operation stages, the scene priority is set from high to low to be an intraoperative stage (remote interactive communication, remote operation), a preoperative stage (start-up self-check, remote pairing, remote connection), and a postoperative stage (remote pairing release, remote pairing switching, shutdown self-check), for different operation tasks, the scene priority is set from high to low to be an operation task, a teaching task and a debugging task, and for different operation types, the scene priority is set from high to low to be an emergency degree.

The method comprises the steps of setting three groups of tele-operation robots in a preoperative stage, an intraoperative stage and a postoperative stage respectively, sequentially reducing the transmission frequency of the tele-operation robots in the postoperative stage according to scene priority, reducing the required bandwidth of the groups of tele-operation robots so that the total required bandwidth is reduced, further reducing the transmission frequency of the tele-operation robots in the preoperative stage if the reduced total required bandwidth is still higher than the current bandwidth, and finally continuing to reduce the transmission frequency of the tele-operation robots in the intraoperative stage by analogy.

In one embodiment, a scene priority is preset for a plurality of surgical scenes, a bandwidth threshold is preset for the network channels, if the total required bandwidth is higher than the current bandwidth in the network channels and is not higher than the bandwidth threshold, the current bandwidth in the network channels is raised to the total required bandwidth, if the total required bandwidth is higher than the current bandwidth in the network channels and the bandwidth threshold, the current bandwidth in the network channels is raised to the bandwidth threshold, at least one group of transmission frequencies of the tele-surgical robots are sequentially lowered according to the scene priority, and the lowered preset required bandwidth corresponding to at least one surgical scene is determined according to the lowered transmission frequency, so that the lowered total required bandwidth is not higher than the bandwidth threshold of the network channels.

The present embodiment further introduces that when the total required bandwidth is higher than the current bandwidth, the current bandwidth of the network channel is preferentially adjusted, and then the transmission frequency of the teleoperation robot is adjusted, so as to adjust the total required bandwidth.

In one embodiment, the interactive data includes multiple data types, multiple types of the interactive data are preset with data type priorities according to different data types, the network channel is preset with a bandwidth threshold, if the total required bandwidth is higher than the current bandwidth in the network channel and the bandwidth threshold, the current bandwidth in the network channel is raised to the bandwidth threshold of the network channel, and according to the data type priorities, the preset required bandwidth corresponding to at least one data type is reduced in each operation scene in sequence, so that the reduced total required bandwidth is stopped when the reduced total required bandwidth is not higher than the bandwidth threshold of the network channel.

The present embodiment further introduces that when the total required bandwidth is higher than the bandwidth threshold, the current bandwidth of the network channel is preferentially adjusted, and then the preset required bandwidth of the teleoperation robot is adjusted so as to adjust the total required bandwidth.

The data type priority comprises, in order from high to low, an endoscope video data type, a master-slave control signaling data type, an energy control data type, an audio data type, an operating room cradle head video data type, an operating room gun camera video data type and a pairing data type, wherein the preset required bandwidth corresponding to one data type in all operation scenes can be reduced each time according to the data type priority from low to high.

For example, the first operation scene includes an endoscope video data type, an audio data type and an operation room rifle bolt video data type, the second operation scene includes an endoscope video data type, a master-slave control signaling data type, an energy control data type, an audio data type and an operation room holder video data type, the third operation scene includes a pairing data type, an audio data type, an operation room holder video data type and an operation room rifle bolt video data type, and specific examples of the data type sequential reduction process are as follows:

firstly reducing the required bandwidth of the paired data types in the third operation scene;

the required bandwidth of the video data types of the operating room cradle head in the second operation scene and the third operation scene is reduced for the second time;

thirdly, reducing the required bandwidth of the gun camera video data types of the operating room in the first operation scene and the third operation scene;

and so on, stopping until the reduced total required bandwidth is not above the bandwidth threshold of the network channel.

In one embodiment, the remote operation robot further comprises an in-vitro image acquisition device which is in communication connection with the first terminal or the second terminal detachably connected with the auxiliary operation device, an operation scene image is acquired through the in-vitro image acquisition device, wherein the operation scene image comprises the auxiliary operation device and an image of the first terminal or the second terminal detachably connected with the auxiliary operation device, and the second control state of the remote operation robot is determined based on the dismounting operation by identifying the dismounting operation of the first terminal or the second terminal and the auxiliary operation device according to the operation scene image.

Further, the center server is in communication connection with a network operator server, and a bandwidth adjustment request is initiated to the network operator server through the center server according to the adjusted current bandwidth in the network channel, so that the network channel is adjusted to be the adjusted current bandwidth through the network operator server, wherein the adjusted current bandwidth comprises the total required bandwidth, the adjusted total required bandwidth or a preset bandwidth threshold of the network channel.

It should be noted that, within the technical scope of the present disclosure, other ordering schemes that can be easily considered by those skilled in the art should also be within the scope of the present disclosure, and are not described in detail herein.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional units or modules according to needs, i.e. the internal structure of the mobile terminal is divided into different functional units or modules to perform all or part of the above-described functions. The functional modules in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the module in the mobile terminal may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

The embodiments of the present application also provide a computer readable storage medium storing a computer program, which when executed by a processor implements steps of the above-described respective method embodiments.

Embodiments of the present application provide a computer program product which, when run on a mobile terminal, causes the mobile terminal to perform steps that enable the implementation of the method embodiments described above.

It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.

As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a condition or event is determined" or "if a condition or event is detected" may be interpreted in the context to mean "upon determination" or "in response to determination" or "upon detection of a condition or event, or" in response to detection of a condition or event.

Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.

Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above embodiment, or may be implemented by a computer program to instruct related hardware, and the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include any entity or device capable of carrying computer program code, recording medium, USB flash disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media, among others. It should be noted that the content of the computer readable medium can be appropriately increased or decreased according to the requirements of the jurisdiction's jurisdiction and the patent practice, for example, in some jurisdictions, the computer readable medium does not include electrical carrier signals and telecommunication signals according to the jurisdiction and the patent practice.

The foregoing embodiments are merely for illustrating the technical solution of the present invention, but not for limiting the same, and although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solution described in the foregoing embodiments may be modified or substituted for some of the technical features thereof, and that these modifications or substitutions should not depart from the spirit and scope of the technical solution of the embodiments of the present invention and should be included in the protection scope of the present invention.

Claims

1. A bandwidth adjustment method for a remote surgical robot, comprising:

A remote surgical robot comprises a first terminal and a second terminal in communication connection, wherein one end of the first terminal and the second terminal is a remote doctor console and the other end is a patient surgical platform, wherein the remote doctor console is provided with a first remote host, and the patient surgical platform is provided with a second remote host, the remote doctor console and the patient surgical platform are respectively connected to an information network channel through the first remote host and the second remote host, and communicate through the information network channel through nodes representing the remote doctor console and the patient surgical platform in a cloud network;

a central server, communicatively connected to at least one of the first terminal and the second terminal, and the central server is communicatively connected to a preset network operator server;

The bandwidth adjustment method comprises:

In response to the interactive operation of the first terminal on the second terminal, triggering a first control state of the remote surgical robot;

According to the first control state, adjusting, by the central server, the bandwidth of the information network channel of the communication connection between the first remote host in the first terminal and the second remote host in the second terminal;

According to the first control state, adjusting the bandwidth of the information network channel of the communication connection between the first remote host in the first terminal and the second remote host in the second terminal by the central server includes:

Matching bandwidth parameters corresponding to the first control state through at least one of a first remote host of the first terminal and a second remote host of the second terminal, and sending a first bandwidth adjustment request to the central server based on the bandwidth parameters, and initiating a second bandwidth adjustment request to the preset network operator server through the central server based on the first bandwidth adjustment request, so as to adjust the bandwidth of the information network channel of the communication connection between the first terminal and the second terminal through the preset network operator server, wherein at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameter includes multiple bandwidth levels in order of bandwidth, and the mapping information includes: mapping relationships between multiple different first control states and multiple bandwidth levels; sending a first bandwidth adjustment request to the central server based on the bandwidth parameters, including: matching a target bandwidth level corresponding to the first control state in the stored mapping information through at least one of the first terminal and the second terminal, and initiating a first bandwidth adjustment request to the central server based on the target bandwidth level; or,

In response to an interactive operation performed between the first terminal or the second terminal and an auxiliary surgical device, the second control state of the remote surgical robot is triggered, and the interactive operation at least includes a disassembly and assembly operation corresponding to the detachable connection; when the first control state and the second control state are triggered simultaneously, one of the first control state and the second control state is selected from the first control state and the second control state according to a preset control state priority through at least one of the first terminal and the second terminal; according to the selected control state, the bandwidth of the communication connection between the first terminal and the second terminal is adjusted through the central server; wherein, at least one of the first terminal and the second terminal is detachably connected to at least one auxiliary surgical device.

2. The bandwidth adjustment method according to claim 1, characterized in that the interactive operation comprises: an operation for triggering any one of a plurality of surgical stages, and an operation for triggering any one of a plurality of surgical tasks;

The first control state indicates: in any surgical stage and/or any surgical task, the state of the interactive operation authority of the first terminal to the second terminal.

3. The bandwidth adjustment method according to claim 2, characterized in that any one of the surgical stages includes at least one of the following: power-on self-test, remote pairing, remote connection, remote interactive communication, remote surgical operation, remote pairing release, and power-off self-test;

Any of the surgical tasks includes at least one of the following: a surgical operation task, a teaching task, and a debugging task.

4. The bandwidth adjustment method according to claim 2, characterized in that the remote physician console and the patient surgery platform include operating components, and the operating components include at least one of the following: a graphical user interface and physical hardware;

In response to the interactive operation of the first terminal on the second terminal, triggering the first control state of the remote surgery robot includes:

In response to a touch operation on a graphical user interface of the remote physician console or a physical operation on physical hardware, triggering an interactive operation on the patient surgical platform;

In response to a touch operation on the graphical user interface of the patient surgery platform or a physical operation on the physical hardware, an interactive operation on the remote doctor console is triggered.

5. The bandwidth adjustment method according to claim 1, characterized in that it further comprises an in vitro image acquisition device, communicatively connected to the first terminal or the second terminal detachably connected to the auxiliary surgery device;

Before triggering the second control state of the remote surgical robot in response to the interactive operation performed by the first terminal or the second terminal and the auxiliary surgery device, the method further includes:

Acquiring a surgical scene image by means of the in vitro image acquisition device, wherein the surgical scene image includes an image of the auxiliary surgical device and a first terminal or a second terminal detachably connected to the auxiliary surgical device;

According to the surgical scene image, the disassembly and assembly operation of the first terminal or the second terminal and the surgical auxiliary device is identified through the first terminal or the second terminal detachably connected to the surgical auxiliary device.

6. The bandwidth adjustment method according to any one of claims 1 to 5, characterized in that the first control state includes: a third control state of the remote doctor console over the patient surgical platform, and a fourth control state of the patient surgical platform over the remote doctor console, and a master-slave state priority is preset between the third control state and the fourth control state;

The central server is communicatively connected to one of the remote doctor console and the patient surgery platform;

According to the first control state, adjusting, by the central server, a bandwidth of a communication connection between the first terminal and the second terminal, comprises:

The third control state is sent to the patient surgery platform through the remote doctor console, and the patient surgery platform selects one of the control states from its current fourth control state and the received third control state according to the master-slave state priority; or,

The fourth control state is sent to the remote doctor console through the patient surgery platform, and the remote doctor console selects one control state from its current third control state and the received fourth control state according to the master-slave state priority;

According to the control state selected by the remote doctor console or the patient surgery platform, the bandwidth of the communication connection between the remote doctor console and the patient surgery platform is adjusted through the central server.

7. The bandwidth adjustment method according to any one of claims 1 to 5, characterized in that the first control state includes: a third control state of the remote doctor console over the patient surgical platform, and a fourth control state of the patient surgical platform over the remote doctor console, and a master-slave state priority is preset between the third control state and the fourth control state;

The central server is respectively connected to the remote doctor console and the patient surgery platform for communication;

sending the third control state to the central server via the remote doctor console, and sending the fourth control state to the central server via the patient surgery platform;

Through the central server, one of the control states is selected from the third control state and the fourth control state according to the master-slave state priority, and the bandwidth of the communication connection between the remote doctor console and the patient surgery platform is adjusted according to the selected control state.

8. A bandwidth adjustment system for a remote surgical robot, using the bandwidth adjustment method for a remote surgical robot according to any one of claims 1 to 7, wherein the bandwidth adjustment system comprises:

The processor is configured as:

Matching bandwidth parameters corresponding to the first control state through at least one of the first remote host and the second remote host, and sending a first bandwidth adjustment request to the central server based on the bandwidth parameters, and initiating a second bandwidth adjustment request to the preset network operator server through the central server based on the first bandwidth adjustment request, so as to adjust the bandwidth of the information network channel of the communication connection between the first terminal and the second terminal through the preset network operator server, wherein at least one of the first terminal and the second terminal stores mapping information, and the bandwidth parameters include multiple bandwidth levels in order of bandwidth, and the mapping information includes: mapping relationships between multiple different first control states and multiple bandwidth levels; sending a first bandwidth adjustment request to the central server based on the bandwidth parameters includes: matching a target bandwidth level corresponding to the first control state in the stored mapping information through at least one of the first terminal and the second terminal, and initiating a first bandwidth adjustment request to the central server based on the target bandwidth level; or,

9. A readable storage medium, characterized in that a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, the steps of the bandwidth adjustment method of the remote surgical robot as described in any one of claims 1 to 7 are implemented.