CN115278160A - Device and method for integrating and displaying surgical robot data - Google Patents

Abstract

The invention relates to a device and a method for integrating and displaying surgical robot data, comprising the following steps: the device comprises a data acquisition unit, a data processing unit, a display unit, an external storage unit and a communication unit; the data acquisition unit comprises a video acquisition unit and an information acquisition unit; the data processing unit comprises a signal conversion unit, an internal storage unit, a video synthesis unit and a backup unit; the signal conversion unit is used for converting the data acquired by the data acquisition unit and the data output by the data processing unit; the video synthesis unit is used for synthesizing the data acquired by the data acquisition unit; the display unit is used for displaying the data processed by the data processing unit; the external storage unit is used for storing the data processed by the data processing unit; the communication unit is used for data communication among the data acquisition unit, the data processing unit, the display unit and the external storage unit, and the communication comprises wireless communication and wired communication.

Description

Device and method for data integration and display of surgical robot

technical field

The invention relates to the technical field of surgical robots, in particular to a device and method for integrated display of surgical robot data.

Background technique

With the rapid development of today's robot technology, especially the development of surgical robot technology, the application scope and demand of surgical robots are also expanding. Surgical robots are stable and precise. Doctors can use surgical robots to complete high-precision surgical actions, reduce the chance of bacterial infection, and improve the quality of surgery. The existing surgical robot equipment is huge, and there are many monitors, which is not conducive to the timely exploration of doctors during the operation; the real-time communication of medical staff is inconvenient when they are in a long distance. Since the endoscopic video of the existing technology is directly connected to the 3D display, and the operation status is connected to another separate display, the disadvantage is that there are too many screens, and the medical staff in the operating room cannot obtain all the information in time; in addition, the existing technology utilizes commercial The video acquisition card sends the endoscope video acquisition to the industrial computer and synthesizes the operation status, and then transmits it to the video acquisition card for display. The disadvantage of this is that the system is large in size and the delay is uncontrollable. Accidents, so time control is very important.

Contents of the invention

The object of the present invention is to propose a device and method for integrating and displaying surgical robot data in view of the prior art.

Specifically, a device for integrating and displaying surgical robot data, including: a data acquisition unit, a data processing unit, a display unit, an external storage unit, and a communication unit;

The data acquisition unit includes a video acquisition unit and an information acquisition unit;

Data processing unit comprises FPGA and DSP; FPGA comprises the first signal conversion unit, internal storage unit, video synthesis unit, backup unit, and DSP comprises the second signal conversion unit, image conversion unit; The first signal conversion unit is used for converting FPGA to receive The data sent by the video acquisition unit and the data output by the FPGA; the second signal conversion unit is used to convert the data sent by the information acquisition unit received by the DSP and the data output by the DSP; the image conversion unit is used to convert the data received by the DSP into image data; The video synthesis unit is used to synthesize the data received by the FPGA; the display unit is used to display the received data processed by the data processing unit, and the display unit includes a local display and/or a remote display;

The external storage unit is used to store the data processed by the data processing unit;

The communication unit is used for data communication between the data acquisition unit, the data processing unit, the display unit and the external storage unit, and the communication includes wireless communication and wired communication.

Further, the video acquisition unit is installed on the endoscope at the operation end for real-time acquisition of video inside the operation object; the information acquisition unit is installed at the control end for real-time acquisition including gesture instructions and/or input instructions. The endoscopes include gastroscopes, colonoscopes, cystoscopes, bronchoscopes, thoracoscopes, laparoscopes, laryngoscopes, nasopharyngoscopes, colposcopes, hysteroscopes and other devices that can directly observe the state of internal tissues and organs of the human body.

Further, the first signal conversion unit is used to convert the signal received by the video acquisition unit into an RGB signal; the second signal conversion unit is used to decode the signal received by the DSP, and encode the signal sent by the DSP; the internal storage unit realizes data storage Temporary storage to achieve high-speed access; the video synthesis unit synthesizes the data collected by the video acquisition unit and the data sent by the DSP; the local display shows the data synthesized by the video synthesis unit; the remote display displays and receives the synthesized video sent by the DSP ; The external storage unit stores the synthesized video sent by the DSP.

Further, the data collected by the video acquisition unit communicates with the FPGA; the data collected by the information acquisition unit communicates with the DSP; the first signal conversion unit converts the received signal of the data collected by the video acquisition unit into an RGB signal; the DSP will receive The command data is processed into image data and sent to PFGA; the image data sent by DSP is stored in the internal storage unit; the video synthesis unit synthesizes the video data stored in the internal storage unit and the image data sent by DSP; the first signal conversion unit Convert the synthesized data into video data in RGB format; the backup unit backs up the video data, transmits the video data to the local display unit for display, and transmits the video data to the DSP; the DSP transmits the synthesized video data to the external storage unit for processing Store and transmit the synthesized video data to the remote display through the communication unit for remote display.

Further, it also includes a voice output unit, the voice output unit is installed on the operation end and the control end or is installed on the operation end and the remote control end, and communicates with the DSP for outputting voice; the information collection unit also includes an audio collection unit, the audio The acquisition unit includes a first audio acquisition unit and a second audio acquisition unit; the communication unit also includes data communication between a data acquisition unit, a data processing unit, a display unit, an external storage unit and a voice output unit, and the communication includes wireless communication and wired communication. communication.

Specifically, a method for integrating and displaying surgical robot data, comprising: the data acquisition unit collects data, and communicates the collected data with the data processing unit through the communication unit; the data processing unit converts the data collected by the data acquisition unit and converts the data Synthesizing and backing up; the data processing unit converts the synthesized data, and communicates with the external storage unit and/or the display unit through the communication unit.

Further, the data collection unit collects data, and communicates the collected data with the data processing unit through the communication unit. The specific steps are: the video collection unit collects video data and sends the collected video data to the FPGA through the communication unit; information collection The unit collects gesture instructions and/or input instructions and sends the collected data to the DSP through the communication unit.

Further, the data processing unit converts the data collected by the data acquisition unit and synthesizes and backs up the data. The specific steps are: the data processing unit receives the video data and data sent to the data processing unit through the communication unit, and the signal conversion unit converts the video data and data Convert to RGB signal data and store in the internal storage unit to realize mass data movement between the internal storage unit and bus-based video processing IP; extract the data in the internal storage unit and synthesize video through the video synthesis unit, and back up the synthesized video .

Further, the video acquisition unit collects video data and sends the collected video data to the FPGA through the communication unit; the information collection unit collects gesture instructions and/or input instructions and sends the collected instructions to the DSP; the FPGA passes the first signal conversion The unit converts the received video data signal into an RGB signal; sends the RGB data to the bus through the Video to AXI4 IP module; stores the RGB data into the internal bus through the VDMA (Video Direct Memory Access) IP module storage unit; realize the large amount of data movement between the storage unit in the system and the video processing IP based on the bus; DSP receives the instruction data collected by the information acquisition unit, and processes the instruction data received by the DSP into image data , and send to PFGA through the GPMC bus; PFGA receives the image data sent by the DSP and stores it in the internal storage unit; extracts the data stored in the internal storage unit through the VDMAIP module, and uses the video synthesis unit to store the video in the internal storage unit The data and the data synthesize new video data; the first signal conversion unit converts the synthesized video into RGB format video data; the backup unit backs up the synthesized new video data, sends it to the local display unit for display, and synthesizes The video data is sent to the DSP; the DSP transmits the synthesized video data to the external storage unit for storage, and transmits the synthesized video data to the remote display through the communication unit for remote display.

Further, the first audio acquisition unit collects the audio data of the operation end in real time, the second audio acquisition unit collects the audio data of the control end in real time, and communicates the collected data with the DSP through the communication unit; the DSP compresses the audio data collected by the first audio acquisition unit. The audio data is sent to the operation end through the communication unit, the operation end voice output unit outputs audio, the DSP decompresses the audio data collected by the second audio acquisition unit and sends it to the remote control end through the communication unit, and the remote control end voice output unit outputs audio; The data processing unit communicates with the external storage unit, the display unit and the voice output unit through the communication unit.

The advantages of the present invention are:

Integrate and display endoscopic video and surgical status information to reduce the number of displays; use chips integrated with FPGA and DSP to synthesize endoscopic video and surgical status information, with high integration, small size, small delay, and reduced Multi-screen observation of medical staff during the operation, the probability of missing important information and causing medical accidents, reducing the medical accidents caused by the delay of intraoperative instructions; audio transmission, to solve the problem of inconvenient information exchange between the control end and the execution end medical staff; Middle audio and video backup, convenient for follow-up accident investigation, teaching function, etc.; solves the problem of remote information transmission; solves the problem of audio and video transmission.

Description of drawings

Fig. 1 is a device for integrating and displaying surgical robot data provided by an embodiment of the present invention;

Fig. 2 is a device for integrating and displaying surgical robot data provided by another embodiment of the present invention;

Fig. 3 is a method for integrating and displaying surgical robot data provided by an embodiment of the present invention;

Fig. 4 is a method for integrating and displaying surgical robot data according to another embodiment of the present invention.

Detailed ways

The technical solutions of the present invention will be described in more detail below in conjunction with the accompanying drawings, and the present invention includes but not limited to the following embodiments.

The endoscope is equipped with a video acquisition unit, which is used to collect the visual field that is not easy to view during the operation, such as the patient's abdominal cavity, the patient's uterus and other places where the visual field is obstructed. The endoscope includes gastroscope, colonoscope, cystoscope, bronchoscope, chest cavity Laryngoscope, Laparoscope, Laryngoscope, Nasopharyngoscope, Colposcope, Hysteroscope, etc.

As shown in accompanying drawing 1, a kind of surgical robot data integrated display device of the present invention, this device comprises: data acquisition unit, data processing unit, display unit, external storage unit, communication unit; Wherein, data acquisition unit comprises video acquisition unit unit, information collection unit;

The data processing unit includes an FPGA and a DSP; the FPGA includes a first signal conversion unit, an internal storage unit, a video synthesis unit, and a backup unit, and the DSP includes a second signal conversion unit and an image conversion unit;

The first signal conversion unit is used to convert the data sent by the video acquisition unit received by the FPGA and the data output by the FPGA; the second signal conversion unit is used to convert the data sent by the information acquisition unit received by the DSP and the DSP The output data; the image conversion unit is used to convert the data received by the DSP into image data; the video synthesis unit is used to synthesize the data received by the FPGA; the display unit is used to display the data processed by the received data processing unit data, the display unit includes a local display and/or a remote display;

The external storage unit is used to store the data processed by the data processing unit, wherein the external storage unit and the DSP are connected through, for example, an IDE interface; the communication unit is used for the data acquisition unit, the data processing unit, the display unit, the Data communication between external storage units, the communication includes wireless communication and wired communication.

Among them, the video acquisition unit is installed on the endoscope at the operation end for real-time acquisition of video inside the operation object during the operation; the information acquisition unit is installed at the control end for real-time acquisition including gesture instructions and/or input instructions. Among them, gesture instructions include instructions issued by gestures such as the motion state of the robotic arm, the pose of the end of the robotic arm, etc., and input instructions include touch screen input, keyboard input, button input, etc. Information such as the position of the end of the manipulator, where the pose of the end of the manipulator includes instructions such as end depth and angle, which are convenient for direct display in the synthesized video, and can accurately determine the displacement position angle and the distance to be displaced during the operation of the manipulator without thinking And angle, reduce the judgment error caused by only video display, and reduce the observation time of the controller watching the video.

Among them, the first signal conversion unit is used to convert the received video acquisition unit and the signal sent by DSP into RGB signal; the second signal conversion unit is used to decode the signal received by DSP, and encode the signal sent by DSP; DSP will receive The described instruction data of described instruction is processed into image data, and send to PFGA, and internal storage unit realizes the temporary storage of data, realizes high-speed access; The signal conversion unit converts the synthesized data into RGB format video data; the backup unit backs up the video data, transmits the video data to the local display unit for display, and transmits the video data to the DSP; the DSP transmits the synthesized video data to the external The storage unit stores and transmits the synthesized video data to the remote display through the communication unit for remote display. Wherein, the local display displays the data synthesized by the video synthesis unit; the remote display displays the synthesized video sent by the receiving DSP; the external storage unit stores the synthesized video sent by the DSP.

Wherein, the data collected by the video acquisition unit communicates with the FPGA, and the communication includes wired and wireless communication, such as communicating through a wired HDMI interface, and converting the video data transmitted by the HDMI interface into RGB signal data. The data collected by the information acquisition unit of the control end is transmitted to the processing end, and the processing end transmits the data to the data processing unit, and communicates with the DSP. The communication includes wired and wireless communication, such as communication through a wired HDMI interface, and the DSP receives the instruction signal transmitted by the processing end. The DSP processes the instruction signal, for example, processes the instruction signal into image information, and sends it to the PFGA. The command signal includes gesture commands such as the operating state of the robotic arm, the pose of the end of the robotic arm, and the pose of the end of the robotic arm includes instructions such as the depth and angle of the end. Wherein, the FPGA converts, for example, the signal of the data collected by the video acquisition unit received by the HDMI interface into an RGB signal through a signal conversion unit such as a DVI to RGB Decoder IP module. Send the RGB data to the bus such as the AXI4 bus through the Video to AXI4 IP module. Through the VDMA (Video Direct Memory Access video register direct access, access) IP module, a large amount of data movement between the internal storage unit and the bus-based video processing IP is realized, and the high-speed access of the data on the bus and the data in the DDR memory is realized. The data processed by the data processing unit sent by the DSP through the GPMC bus IP, that is, the data processed into image information is stored in the internal storage unit, wherein the internal storage unit is used to temporarily store computing data, and the internal storage unit is such as DDR memory. The video synthesis unit synthesizes the video data stored in the internal storage unit and the image data sent by the DSP; the signal conversion unit such as the AXI4 to Video IP module converts the synthesized data into RGB format video data; the backup unit backs up the video data. The FPGA transmits the synthesized video data to the local display unit for local display, and transmits the video data to the DSP; the DSP transmits the synthesized video data to the external storage unit for storage, and transmits the synthesized video data to the remote through the communication unit Remote display on the terminal monitor is convenient for remote medical staff to view in real time, quickly obtain data information, and make timely judgments.

Among them, the surgical robot includes a surgical end, a processing end, and a control end. As shown in Figure 2, another device for integrating and displaying surgical robot data according to the present invention includes: a data acquisition unit, a data processing unit, a display unit, an external storage unit, a communication unit, and a voice output unit. Wherein, the voice output unit is installed on the operation end and the control end or is installed on the operation end and the remote control end, and communicates with the data processing unit for outputting voice; the information collection unit also includes an audio collection unit; the communication unit is used for The data communication between the data collection unit, the data processing unit, the display unit, the external storage unit and the voice output unit, the communication includes wireless communication and wired communication. The data acquisition unit includes a video acquisition unit, an information acquisition unit, and an audio acquisition unit; the data processing unit includes a signal conversion unit, an internal storage unit, a video synthesis unit, and a backup unit; the signal conversion unit is used to convert the data and data collected by the data acquisition unit For the data output by the processing unit, the video synthesis unit is used to synthesize the data collected by the data acquisition unit; the display unit is used to display the data processed by the data processing unit, including a local display and/or a remote display; the external storage unit is used for Store the data processed by the data processing unit, wherein the external storage unit and DSP are connected through, for example, an IDE interface; the communication unit is used for data communication between the data acquisition unit, data processing unit, display unit, external storage unit and audio output unit, so The communication mentioned above includes wireless communication and wired communication.

Among them, the video acquisition unit is installed on the endoscope at the operation end to collect the video inside the operation object during the operation; the information acquisition unit is installed on the control end to collect gesture instructions and/or input instructions; the audio acquisition unit is used for Collect audio data. Among them, gesture instructions include instructions issued by gestures such as the motion state of the robotic arm, the pose of the end of the robotic arm, etc., and input instructions include touch screen input, keyboard input, button input, etc. Information such as the position of the end of the manipulator, where the pose of the end of the manipulator includes instructions such as end depth and angle, which are convenient for direct display in the synthesized video, and can accurately determine the displacement position angle and the distance to be displaced during the operation of the manipulator without thinking And angle, reduce the judgment error caused by only video display, and reduce the observation time of the controller watching the video.

Wherein, the signal conversion unit is used to convert the received signals collected by the video acquisition unit and the information acquisition unit into RGB signals; the internal storage unit realizes temporary storage of data and realizes high-speed access; the video synthesis unit includes synthesizing the received The data collected by the video collection unit and the data collected by the information collection unit; the display unit displays the data synthesized by the video synthesis unit; the voice output unit outputs audio data, and the external storage unit stores the data and audio data synthesized by the video synthesis unit .

Wherein, the data processing unit includes FPGA and DSP. The data collected by the video acquisition unit communicates with the FPGA, and the communication includes wired and wireless communication, such as communication through a wired HDMI interface, and the video data transmitted by the HDMI interface is converted into RGB signal data.

The data collected by the information acquisition unit at the control end is transmitted to the processing end, and the processing end transmits the data to the data processing unit to communicate with the DSP. The command signal and voice signal transmitted by the processing terminal. The DSP processes the instruction signal, for example, processes the instruction signal into image information, and sends it to the PFGA. At the same time, the DSP encodes and decodes the audio data, outputs the audio data through the voice output unit, and stores the audio data in the external storage unit. The command signals include gesture command signals and input command signals, where the command signals include gesture commands such as the operation status of the manipulator, the pose of the end of the manipulator, and the pose of the end of the manipulator includes commands such as end depth and angle. The input instructions include touch screen input, keyboard input, button input, etc., which are issued by input, specifically including: operation type, appliance type, position of the end of the robotic arm, etc., where the pose of the end of the robotic arm includes commands such as end depth and angle.

Wherein, the FPGA converts, for example, the signal of the data collected by the video acquisition unit received by the HDMI interface into an RGB signal through a signal conversion unit such as a DVI to RGB Decoder IP module. Send the RGB data to the bus such as the AXI4 bus through the Video to AXI4 IP module. Through the VDMA (Video Direct Memory Access video register direct access, access) IP module, a large amount of data movement between the internal storage unit and the bus-based video processing IP is realized, and the high-speed access of the data on the bus and the data in the DDR memory is realized. The data processed by the data processing unit sent by the DSP through the GPMC bus IP, that is, the data processed into image information is stored in the internal storage unit, wherein the internal storage unit is used to temporarily store computing data, and the internal storage unit is such as DDR memory. The video synthesis unit synthesizes the video data stored in the internal storage unit and the image data sent by the DSP; the signal conversion unit such as the AXI4 to Video IP module converts the synthesized data into RGB format video data; the backup unit backs up the video data; FPGA converts the synthesized The video data is transmitted to the local display unit for local display, and the video data is transmitted to the DSP; the DSP transmits the synthesized video data and voice data to the external storage unit for storage, and encodes the synthesized video data and voice data, through The communication unit is transmitted to the remote display for remote display, which is convenient for remote medical staff to view and communicate in real time, quickly obtain data information, make timely judgments, and provide timely guidance for operations, reducing the probability and frequency of medical accidents during operations.

Among them, the surgical robot includes a surgical terminal, a processing terminal, a control terminal and a remote control terminal. Wherein, the local display is set on the control end, for example, and the remote display is set on the remote control end, so as to facilitate timely and simultaneous display of the synthetic data of the local display and the synthetic data of the remote display.

As shown in accompanying drawing 3, a kind of surgical robot data integrated display method of the present invention, this method comprises the following steps:

S110: The data collection unit collects data, and communicates the collected data with the data processing unit through the communication unit. The specific steps are: the video acquisition unit collects video data and sends the collected video data to the data processing unit through the communication unit; the information collection unit collects gesture instructions and/or input instructions and sends the collected data to the data processing unit through the communication unit unit.

S120: The data processing unit converts the data collected by the data acquisition unit and synthesizes and backs up the data. The specific steps are: the data processing unit receives the video data and data sent to the data processing unit through the communication unit, the signal conversion unit converts the video data and data into RGB signal data, and stores them in the internal storage unit to realize the internal storage unit and the bus-based video Handle a large amount of data movement between IPs; extract the data in the internal storage unit and synthesize video through the video synthesis unit, and back up the synthesized video.

Wherein the data processing unit includes FPGA and DSP, and the specific steps include:

The video acquisition unit collects video data and sends the collected video data to the FPGA through the communication unit; the information collection unit collects gesture instructions and/or key-in instructions and sends the collected instructions to the processing terminal, and the processing terminal transmits the data to The data processing unit communicates with the DSP. The FPGA converts the video data signal received by the HDMI interface into an RGB signal through a signal conversion unit such as the DVI to RGB DecoderIP module; sends the RGB data to the bus such as the AXI4 bus through the Video to AXI4 IP module; through the VDMA (Video Direct Memory Access Video register direct access, access) IP module stores RGB data in the internal storage unit; realizes a large amount of data movement between the storage unit in the system and the video processing IP based on the bus; accesses the data on the bus such as the AXI4 bus and the DDR memory high-speed access between The DSP receives instruction data, and processes the instruction data received by the DSP, such as gesture instructions and/or input instructions, into image data, and sends them to the PFGA through a bus, such as a GPMC bus.

Among them, gesture instructions include instructions issued by gestures such as the motion state of the robotic arm, the pose of the end of the robotic arm, etc., and input instructions include touch screen input, keyboard input, button input, etc. Information such as the position of the end of the manipulator, where the pose of the end of the manipulator includes instructions such as end depth and angle, which are convenient for direct display in the synthesized video, and can accurately determine the displacement position angle and the distance to be displaced during the operation of the manipulator without thinking And angle, reduce the judgment error caused by only video display, and reduce the observation time of the controller watching the video.

PFGA receives the image data sent by DSP and stores it in the internal storage unit such as DDR memory; extracts the video data and data stored in the internal storage unit through the VDMAIP module, and synthesizes new video data through the video synthesis unit; the signal conversion unit such as AXI4 to Video IP converts the synthesized video into RGB format video data; the backup unit backs up the synthesized new video data, PFGA sends the synthesized video data to the local display unit for display, and sends the synthesized new video data to the DSP .

S130: The data processing unit converts the synthesized data, and communicates with the external storage unit and/or the display unit through the communication unit. Specifically, the DSP receives the synthesized new video data sent by the backup unit, transmits the synthesized video data to the external storage unit for storage, and transmits the synthesized video data to the remote display through the communication unit for remote display.

As shown in Figure 4, another method for integrating and displaying surgical robot data according to the present invention includes the following steps:

S210: The data collection unit collects data, and communicates the collected data with the data processing unit through the communication unit.

The specific steps are: the video acquisition unit collects video data and sends the collected video data to the data processing unit through the communication unit; the information collection unit collects gesture instructions and/or input instructions and sends the collected data to the data processing unit through the communication unit unit, the information collection unit also includes an audio collection unit, the audio collection unit collects the operation end and the control end or the operation end and the remote control end, the audio collection unit collects the voice of the medical staff at the operation end and/or the voice of the medical staff at the control end, or the audio collection unit Collect the voice of medical staff at the operation end and/or the voice of medical staff at the remote control end to facilitate communication between medical staff, ensure real-time sharing of information, and reduce surgical failures. Wherein, the voice output unit is installed on the operation end and the control end or is installed on the operation end and the remote control end, and communicates with the data processing unit for outputting voice; the information collection unit also includes an audio collection unit; the communication unit is used for The data communication between the data collection unit, the data processing unit, the display unit, the external storage unit and the voice output unit, the communication includes wireless communication and wired communication.

S220: The data processing unit converts the data collected by the data acquisition unit and synthesizes, backs up and encodes the data.

The specific steps are: the data processing unit receives the video data, audio data and data sent to the data processing unit through the communication unit, the signal conversion unit converts the video data and data into RGB signal data, and stores them in the internal storage unit to realize the internal storage unit and based on The video of the bus handles a large amount of data movement between IPs; extracts the data in the internal storage unit and synthesizes video through the video synthesis unit, backs up the synthesized video, and encodes the backup synthesized video and voice. Wherein the data processing unit includes FPGA and DSP, and the specific steps include:

The video acquisition unit collects video data and sends the collected video data to the FPGA through the communication unit; the information collection unit collects gesture instructions and/or key-in instructions and sends the collected instructions to the processing terminal, and the processing terminal transmits the data to The data processing unit communicates with the DSP. The FPGA converts the video data signal received by the HDMI interface into an RGB signal through a signal conversion unit such as the DVI to RGB DecoderIP module; sends the RGB data to the bus such as the AXI4 bus through the Video to AXI4 IP module; through the VDMA (Video Direct Memory Access Video register direct access, access) IP module stores RGB data in the internal storage unit; realizes a large amount of data movement between the storage unit in the system and the video processing IP based on the bus; accesses the data on the bus such as the AXI4 bus and the DDR memory high-speed access between The DSP receives instruction data, processes the instruction data received by the DSP, such as gesture instructions and/or input instructions, into image data, decodes the received audio data, and sends it to an external storage unit for storage. The DSP sends the image data to the PFGA through a bus such as a GPMC bus.

Among them, gesture instructions include instructions issued by gestures such as the motion state of the robotic arm, the pose of the end of the robotic arm, etc., and input instructions include touch screen input, keyboard input, button input, etc. Information such as the position of the end of the manipulator, where the pose of the end of the manipulator includes instructions such as end depth and angle, which are convenient for direct display in the synthesized video, and can accurately determine the displacement position angle and the distance to be displaced during the operation of the manipulator without thinking And angle, reduce the judgment error caused by only video display, and reduce the observation time of the controller watching the video.

PFGA receives the image data sent by DSP and stores it in the internal storage unit such as DDR memory; PFGA extracts the video data and data stored in the internal storage unit through the VDMAIP module, and synthesizes new video data through the video synthesis unit; the signal conversion unit such as AXI4 to Video IP converts the synthesized video to RGB format video data, the backup unit backs up the synthesized new video data, PFGA sends the synthesized video data to the local display unit for display, and sends the synthesized new video data to the DSP . The DSP transmits the synthesized video data and voice data to the external storage unit for storage, encodes the synthesized video data and voice data, and transmits the synthesized video data and voice data to the remote display for remote display through the communication unit, which is convenient for remote medical staff to view in real time Real-time communication, quick access to data information, timely judgment, and timely guidance to operations, reducing the probability and frequency of medical accidents during operations.

Wherein, the audio collection unit includes a first audio collection unit and a second audio collection unit, the first audio collection unit collects the audio data of the operation end in real time, the second audio collection unit collects the audio data of the control end in real time, and transmits the collected data through communication The unit communicates with the DSP; the DSP compresses the audio data collected by the first audio acquisition unit and sends it to the operation end through the communication unit, the operation end voice output unit outputs audio, and the DSP decompresses the audio data collected by the second audio acquisition unit and sends it through the communication unit Send to the remote control terminal, and the voice output unit of the remote control terminal outputs audio. The data processing unit communicates with the external storage unit, the display unit and the voice output unit through the communication unit.

S230: The data processing unit converts the synthesized data, and communicates with the external storage unit, the display unit and the voice output unit through the communication unit. Specifically include:

The DSP combines the audio data and the synthesized video data, and sends them to the external storage unit for data storage, and the PFGA sends the synthesized video to the display unit, and the display unit displays the synthesized video. The PFGA sends the synthesized video data to the local display unit for display, and sends the synthesized new video data to the DSP. The DSP transmits the synthesized video data to the external storage unit for storage, encodes the synthesized video data and voice data, transmits them to the remote control terminal through the communication unit, and performs remote display on the remote display at the remote control terminal and controls them remotely. The voice output unit at the end performs voice output. The voice acquisition unit collects the voice data of the remote control terminal, transmits it to the control terminal through the communication unit, and the DSP of the control terminal decodes, stores the voice data, and performs voice output. It is convenient for remote medical staff to view and communicate in real time, quickly obtain data information, make timely judgments, and guide operations in a timely manner, reducing the probability and frequency of medical accidents during operations. Among them, the surgical robot includes a surgical terminal, a processing terminal, a control terminal and a remote control terminal. Wherein, the local display is set on the control end, for example, and the remote display is set on the remote control end, so as to facilitate timely and simultaneous display of the synthetic data of the local display and the synthetic data of the remote display.

The present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can implement the present invention by using other various specific embodiments according to the embodiments and the accompanying drawings. The design of the transformation or modification all falls into the protection scope of the present invention.

Claims (11)

1. A device for integrating and displaying surgical robot data is characterized by comprising: the device comprises a data acquisition unit, a data processing unit, a display unit, an external storage unit and a communication unit;

the data acquisition unit comprises a video acquisition unit and an information acquisition unit;

the data processing unit comprises an FPGA and a DSP; the FPGA comprises a first signal conversion unit, an internal storage unit, a video synthesis unit and a backup unit, and the DSP comprises a second signal conversion unit and an image conversion unit;

the first signal conversion unit is used for converting the data received by the FPGA and sent by the video acquisition unit and the data output by the FPGA; the second signal conversion unit is used for converting the data received by the DSP and sent by the information acquisition unit and the data output by the DSP; the image conversion unit is used for converting the data received by the DSP into image data; the video synthesis unit is used for synthesizing data received by the FPGA; the display unit is used for displaying the received data processed by the data processing unit and comprises a local display and/or a remote display;

the external storage unit is used for storing the data processed by the data processing unit;

the communication unit is used for data communication among the data acquisition unit, the data processing unit, the display unit and the external storage unit, and the communication comprises wireless communication and wired communication.

2. The device for integrating and displaying the data of the surgical robot as claimed in claim 1, wherein the video acquisition unit is installed on an endoscope at a surgical end and is used for acquiring the video inside the operation object in real time; the information acquisition unit is arranged at the control end and is used for acquiring gesture instructions and/or key-in instructions in real time.

3. The device for integrating and displaying the data of the surgical robot according to claim 1, wherein the first signal conversion unit is configured to convert the received signal of the video capture unit into RGB signals; the second signal conversion unit is used for decoding the signals received by the DSP and encoding the signals sent by the DSP; the internal storage unit realizes temporary storage of data and high-speed access; the video synthesis unit synthesizes the received data collected by the video collection unit and the data sent by the DSP; the local display displays the data synthesized by the video synthesis unit; the remote display displays and receives the synthesized video sent by the DSP; and the external storage unit stores the synthesized video sent by the DSP.

4. The device for the integrated display of the surgical robot data according to claim 1, wherein the data collected by the video collection unit is communicated with the FPGA; the data acquired by the information acquisition unit is communicated with the DSP; the first signal conversion unit converts the received data signal acquired by the video acquisition unit into an RGB signal; the DSP processes the received data into image data and sends the image data to the PFGA; storing the image data sent by the DSP in an internal storage unit; the video synthesis unit synthesizes the video data stored in the internal storage unit and the image data sent by the DSP; the first signal conversion unit converts the synthesized data into video data in an RGB format; the backup unit backs up video data, transmits the video data to a local display unit for display and transmits the video data to the DSP; and the DSP transmits the synthesized video data to an external storage unit for storage, and transmits the synthesized video data to a far-end display for far-end display through a communication unit.

5. The device for integrating and displaying the data of the surgical robot according to any one of claims 1 to 4, further comprising a voice output unit, wherein the voice output unit is installed at the surgical end and the control end or at the surgical end and the remote control end, is in communication with the DSP, and is used for outputting voice; the information acquisition unit further comprises an audio acquisition unit, and the audio acquisition unit comprises a first audio acquisition unit and a second audio acquisition unit; the communication unit also comprises a data communication unit used for data acquisition, a data processing unit, a display unit, an external storage unit and a voice output unit, and the communication comprises wireless communication and wired communication.

6. A method for integrating and displaying surgical robot data by using the device according to any one of claims 1-5, comprising:

the data acquisition unit acquires data and communicates the acquired data with the data processing unit through the communication unit;

the data processing unit converts the data acquired by the data acquisition unit, synthesizes and backs up the data;

the data processing unit converts the synthesized data and communicates with the external storage unit and/or the display unit through the communication unit.

7. The method for integrating and displaying the surgical robot data according to claim 6, wherein the data acquisition unit acquires data and the step of communicating the acquired data with the data processing unit through the communication unit comprises the following specific steps:

the video acquisition unit acquires video data and sends the acquired video data to the FPGA through the communication unit; the information acquisition unit acquires gesture instructions and/or key instructions and sends acquired data to the DSP through the communication unit.

8. The method for integrated display of surgical robot data according to claim 6, wherein the data processing unit converts the data collected by the data collecting unit and synthesizes and backs up the data by the specific steps of:

the data processing unit receives video data and data transmitted to the data processing unit through the communication unit, the signal conversion unit converts the video data and data into RGB signal data, and storing the data into an internal storage unit to realize mass data movement between the internal storage unit and a video processing IP based on a bus; and extracting the data in the internal storage unit, synthesizing the video through the video synthesis unit, and backing up the synthesized video.

9. The method for integrating and displaying the surgical robot data according to claim 8, further comprising the step of converting the received video data signal into RGB signal by the FPGA through the first signal conversion unit; sending the RGB data to a bus through a Video to AXI4 IP module; storing the RGB data into a Memory unit through a VDMA (Video Direct Memory Access) IP module; the method realizes the mass data movement between the storage unit in the system and the video processing IP based on the bus; the DSP receives the data acquired by the information acquisition unit, processes the data received by the DSP into image data and sends the image data to the PFGA through a GPMC bus; the PFGA receives the image data sent by the DSP and stores the image data in an internal storage unit; and extracting the data stored in the internal storage unit through the VDMAIP module, and synthesizing the video data and the data stored in the internal storage unit into new video data through the video synthesizing unit.

10. The method for integrated display of surgical robot data according to claim 8, further comprising the steps of converting the synthesized video into video data in RGB format by the first signal conversion unit; the backup unit backs up the synthesized new video data, sends the synthesized new video data to the local display unit for displaying, and sends the synthesized video data to the DSP; and the DSP transmits the synthesized video data to an external storage unit for storage, and transmits the synthesized video data to a remote display for remote display through a communication unit.

11. The method for integrated display of surgical robot data according to claim 6, further comprising the steps of:

the first audio acquisition unit acquires audio data of an operation end in real time, and the second audio acquisition unit acquires audio data of a control end in real time and communicates the acquired data with the DSP through the communication unit;

the DSP compresses the audio data collected by the first audio collection unit and sends the audio data to the operation end through the communication unit, the operation end voice output unit outputs audio, the DSP decompresses the audio data collected by the second audio collection unit and sends the audio data to the remote control end through the communication unit, and the remote control end voice output unit outputs audio;

the data processing unit is communicated with the external storage unit, the display unit and the voice output unit through the communication unit.

Images