CN118216867A - A 3D electronic laparoscope and its image processing method, medium and product - Google Patents

Abstract

The invention discloses a 3D electronic laparoscope, an image processing method, a medium and a product thereof, relating to the field of medical appliances, comprising the following steps: the distance sensor is arranged at the head end part of the image acquisition module; the distance sensor is connected with the image processing module; the distance sensor is used for acquiring the distance between the image acquisition module and the detected tissue; the image acquisition module is connected with the image processing module; the image acquisition module is used for simultaneously acquiring a first real-time image and a second real-time image of the detected tissue; the image processing module is connected with the display module; the image processing module is used for determining horizontal parallaxes of the first real-time image and the second real-time image according to the left-right eye parallax relation comparison table and the distance, and adjusting the horizontal parallaxes. The invention can improve the 3D visual effect of the electronic pleuroperitoneal cavity under different working distances.

Description

A 3D electronic laparoscope and its image processing method, medium and product

Technical Field

The present invention relates to the field of medical devices, and in particular to a 3D electronic laparoscope and an image processing method, medium and product thereof.

Background technique

Endoscope is a commonly used medical device that can be used in various examinations and surgical operations. Compared with traditional surgical operations, the functional minimally invasive surgery of medical endoscopes has been widely accepted by doctors and patients. Medical endoscopes use natural holes in the human body or make small holes when necessary. As long as the doctor skillfully inserts the endoscope lens deep into the body, he can perform closed surgical operations in the body outside the body.

3D electronic thoraco-laparoscopic technology improves doctors' perception of depth, which is impossible with two-dimensional visual effects. In traditional thoraco-laparoscopic surgery, the lack of perception of the subject, depth and layer is the biggest problem that plagues surgeons. The 3D electronic thoraco-laparoscopic system restores the true three-dimensional surgical field of view and the most accurate spatial positioning, while providing the maximum depth and three-dimensional layer of anatomy, improving the accuracy of difficult and complex surgeries and reducing operational risks.

The 3D visual effect of 3D electronic thoracoscopic and laparoscopic instruments directly affects the accuracy of the doctor's intraoperative operation and the comfort of the senses. Inappropriate 3D visual effects will make the doctor feel dizzy when using it. However, 3D electronic thoracoscopic and laparoscopic instruments need to match different 3D visual effects at different working distances, so there is an urgent need for a 3D electronic laparoscope that can automatically adjust the 3D visual effect.

Summary of the invention

The purpose of the present invention is to provide a 3D electronic laparoscope and an image processing method, medium and product thereof, which can improve the 3D visual effect of the electronic thoracoabdominal scope at different working distances.

To achieve the above object, the present invention provides the following solutions:

A 3D electronic laparoscope, comprising a distance sensor, an image acquisition module, an image processing module and a display module;

The distance sensor is arranged at the head end of the image acquisition module; the distance sensor is connected to the image processing module; the distance sensor is used to collect the distance between the image acquisition module and the detected tissue, and send the distance to the image processing module;

The image acquisition module is connected to the image processing module; the image acquisition module is used to simultaneously acquire a first real-time image and a second real-time image of the detected tissue, and send the first real-time image and the second real-time image to the image processing module;

The image processing module is connected to the display module; the image processing module is used to determine the horizontal parallax of the first real-time image and the second real-time image according to the left-right eye parallax relationship comparison table and the distance, and adjust the horizontal parallax according to the observer's observation distance, pupil distance, pupil diameter and unit pixel width of the display module; wherein the left-right eye parallax relationship comparison table is a table of shooting distance and left-right eye parallax relationship.

Optionally, the 3D electronic laparoscope further includes a transmission module; the image processing module is connected to the distance sensor and the image acquisition module respectively through the transmission module.

Optionally, the image processing module is a computer.

A 3D electronic laparoscope image processing method is applied to the above-mentioned 3D electronic laparoscope, and the image processing method comprises:

Acquire a distance, a first real-time image, and a second real-time image; the distance is the distance between the image acquisition module and the detected tissue; the first real-time image and the second real-time image are both acquired simultaneously by the image acquisition module;

Determine the horizontal disparity between the first real-time image and the second real-time image according to the distance and left-right eye disparity relationship comparison table;

Determining whether the horizontal parallax is within a preset range;

When the horizontal parallax is not within a preset range, the distance between the first real-time image and the second real-time image is adjusted.

Optionally, the preset range is:

Wherein, disp _v is the horizontal parallax between the first real-time image and the second real-time image; δ is a constant, d _v represents the distance between the observer and the display, d _e represents the distance between the pupils of the two eyes of a person, D represents the pupil diameter of the human eye, and P _w represents the unit pixel width of the display.

Optionally, when the horizontal parallax is not within a preset range, adjusting the distance between the first real-time image and the second real-time image specifically includes:

when When the first real-time image moves away from the second real-time image The second real-time image moves in a direction away from the first real-time image. Pixels;

when When the first real-time image moves toward the direction close to the second real-time image The second real-time image moves toward a direction close to the first real-time image.

Optionally, the process of acquiring the left-eye and right-eye parallax relationship comparison table specifically includes:

Acquire a first real-time image sample and a second real-time image sample acquired by an image acquisition module at multiple shooting distances;

Using a display module to display the first real-time image sample and the second real-time image sample to obtain left-eye and right-eye parallax corresponding to each shooting distance;

A left-right eye parallax relationship comparison table is constructed according to the left-right eye parallax corresponding to each shooting distance.

A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned 3D electronic laparoscope image processing methods.

A computer program product comprises a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned 3D electronic laparoscope image processing methods.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The present invention uses a distance sensor to collect the distance between the image acquisition module and the detected tissue, and sends the distance to the image processing module. The image acquisition module simultaneously collects a first real-time image and a second real-time image of the detected tissue, and sends the first real-time image and the second real-time image to the image processing module; the image processing module determines the horizontal parallax of the first real-time image and the second real-time image according to a left-right eye parallax relationship comparison table and the distance, and adjusts the horizontal parallax according to the observer's observation distance, pupil distance, pupil diameter and unit pixel width of the display module, so that the present invention can improve the 3D visual effect of the electronic thoracoabdominal scope at different working distances.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic diagram of the appearance of a distance sensor provided in Embodiment 1 of the present invention;

FIG2 is a schematic diagram of the internal connection relationship of the 3D electronic laparoscope provided in Example 1 of the present invention;

FIG3 is a diagram showing the internal structure of a computer device;

FIG4 is a schematic diagram showing the comparison of the left and right eye parallax relationship provided by Example 1 of the present invention;

FIG5 is a schematic diagram of the relationship between display imaging and horizontal parallax provided by Embodiment 1 of the present invention;

FIG. 6 is a schematic diagram of the real-time image movement principle provided by Embodiment 1 of the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The purpose of the present invention is to provide a 3D electronic laparoscope and its image processing method, medium and product, which can improve the 3D visual effect of the electronic thoracoabdominal scope at different working distances, and further, realize that the 3D electronic thoracoabdominal scope can present suitable 3D visual effect at different working distances.

In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

Example 1

As shown in FIG. 1 , a 3D electronic laparoscope in this embodiment includes a distance sensor, an image acquisition module, an image processing module and a display module.

The distance sensor is arranged at the head end of the image acquisition module; the distance sensor is connected to the image processing module; the distance sensor is used to acquire the distance between the image acquisition module and the detected tissue, and send the distance to the image processing module.

The image acquisition module is connected to the image processing module; the image acquisition module is used to simultaneously acquire a first real-time image and a second real-time image of the detected tissue, and send the first real-time image and the second real-time image to the image processing module.

The image processing module is connected to the display module; the image processing module is used to determine the horizontal parallax of the first real-time image and the second real-time image according to the left-right eye parallax relationship comparison table and the distance, and adjust the horizontal parallax according to the observer's observation distance, pupil distance, pupil diameter and unit pixel width of the display module; wherein the left-right eye parallax relationship comparison table is a table of shooting distance and left-right eye parallax relationship.

As a specific implementation, the 3D electronic laparoscope further includes a transmission module; the image processing module is connected to the distance sensor and the image acquisition module respectively through the transmission module.

Furthermore, the image processing module is a computer.

In practical applications, as shown in Figures 1 and 2, the 3D electronic laparoscope includes a distance sensor, a 3D electronic laparoscope image acquisition module (i.e., image acquisition module), a transmission module, a 3D image processor host (i.e., image processing module), and a 3D display (display module). The distance sensor is a mature electronic product; the 3D electronic laparoscope image acquisition module is composed of an optical lens and two CMOS chips. The image of the object being detected is transmitted to the target surface of the two CMOS chips through the optical lens. The two CMOS chips are responsible for processing the optical images of the left and right eyes into electrical signals. The transmission module is composed of a printed circuit board and an electronic signal line, which transmits the electrical signal output by the CMOS chip to the image processing module. The image processing module is composed of a printed circuit board with image processing capabilities, which can process the electrical signal transmitted from the image acquisition module according to the algorithm formula and output it through the video interface.

The distance sensor is located at the head end of the 3D electronic abdominal image acquisition module, and is used to detect the distance between the 3D electronic abdominal image acquisition module and the detected tissue, and transmit the data to the 3D image processor host through the transmission module. The 3D electronic laparoscopic image acquisition module can acquire real-time images of the detected tissue, and transmit them to the 3D image processor host through the transmission module. The 3D image processor host performs 3D visual correction and fusion on the real-time images based on the distance between the acquisition module and the detected tissue, and outputs 3D images, which are displayed on the 3D display. The observer can wear 3D glasses to observe the 3D images of the detected tissue. In addition, the 3D image processor host can display the 2D images of the left and right eyes on the 3D display respectively.

Example 2

On the basis of Example 1, the present invention further provides an image processing method for a 3D electronic laparoscope, the image processing method comprising:

Step S1: Acquire the distance, the first real-time image and the second real-time image; the distance is the distance between the image acquisition module and the detected tissue; the first real-time image and the second real-time image are both acquired simultaneously by the image acquisition module.

Step S2: determining the horizontal disparity between the first real-time image and the second real-time image according to the distance and left-right eye disparity relationship comparison table.

Specifically, the process of obtaining the left-right eye parallax relationship comparison table specifically includes:

Acquire first real-time image samples and second real-time image samples acquired by the image acquisition module at multiple shooting distances.

A display module is applied to display the first real-time image sample and the second real-time image sample to obtain left-eye and right-eye parallaxes corresponding to the shooting distances.

A left-right eye parallax relationship comparison table is constructed according to the left-right eye parallax corresponding to each shooting distance.

Step S3: Determine whether the horizontal disparity is within a preset range.

Specifically, the preset range is:

Wherein, disp _v is the horizontal parallax between the first real-time image and the second real-time image; δ is a constant, d _v represents the distance between the observer and the display, d _e represents the distance between the pupils of the two eyes of a person, D represents the pupil diameter of the human eye, and P _w represents the unit pixel width of the display.

Step S4: When the horizontal parallax is not within a preset range, adjusting the distance between the first real-time image and the second real-time image.

Specifically, when the horizontal parallax is not within a preset range, adjusting the distance between the first real-time image and the second real-time image specifically includes:

when When the first real-time image moves away from the second real-time image The second real-time image moves in a direction away from the first real-time image. Pixel.

when When the first real-time image moves toward the direction close to the second real-time image The second real-time image moves toward a direction close to the first real-time image.

In practical applications, first, the 3D electronic thoracoabdominal scope is placed at different working distances, and the images of the left and right eyes are displayed separately through the 3D display to obtain a comparison table of the parallax relationship between the left and right eyes at different working distances, as shown in FIG4 .

Then, the distance _do between the lens and the photographed object can be directly obtained through the distance sensor, and the horizontal disparity value disp _v between the images photographed by the two cameras can be obtained through the comparison table obtained by the test.

Furthermore, considering the human eye's requirements for 3D imaging effects, the parallax is Within the range, that is Among them, δ = 2.907 × 10 ⁴ rad is a constant, d _v represents the distance between the observer and the display, which is set as a constant and the observer will be informed of the optimal observation distance during the demonstration, d _e represents the distance between the pupils of the human eye, D represents the pupil diameter of the human eye, and P _w represents the unit pixel width of the display, which is also a fixed value. Therefore, the optimal 3D imaging observation range of the human eye is set to a custom range, and a better observation effect can be obtained by simply adjusting the parallax of the left and right images within this range.

If the horizontal disparity value disp _v is not within the range, the image needs to be moved to achieve the effect of reducing the horizontal disparity. As shown in FIG5 , the imaging effect deviates too far from the display, which will cause an uncomfortable feeling. Therefore, the left image of the image needs to be shifted to the right and the right image needs to be shifted to the left.

Furthermore, if Move the left image to the left /> Move right image to the right /> Pixels; if /> Move the left image to the right /> Move the right image to the left /> Therefore, the horizontal parallax of the two images is controlled at/> range to achieve the best 3D imaging effect.

As shown in Figure 6, the left side of the left image and the right side of the right image are cropped respectively. For the fixed point P, the pixel positions in the left and right images achieve the effect of right and left shifting, and then the right and left sides of the cropped image are filled respectively to restore the image to the size before cropping, so that the two images can be merged.

The advantages of the present invention are as follows:

1. Real-time automatic detection of the working distance of the 3D electronic thoracoscopic and laparoscopic device.

2. Automatically adjust the 3D visual effect according to the working distance detected in real time.

Example 3

A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the 3D electronic laparoscope image processing method in Example 1.

Example 4

A computer program product includes a computer program, which implements the steps of the 3D electronic laparoscope image processing method in embodiment 1 when executed by a processor.

Example 5

A computer device, which can be a database, and its internal structure diagram can be shown in Figure 3. The computer device includes a processor, a memory, an input/output interface (Input/Output, referred to as I/O) and a communication interface. Among them, the processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input/output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the image processing method of the 3D electronic laparoscope in Example 2 is implemented.

It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present invention may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided by the present invention may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims (9)

1. A 3D electronic laparoscope, which is characterized by comprising a distance sensor, an image acquisition module, an image processing module and a display module;

The distance sensor is arranged at the head end part of the image acquisition module; the distance sensor is connected with the image processing module; the distance sensor is used for acquiring the distance between the image acquisition module and the detected tissue and sending the distance to the image processing module;

the image acquisition module is connected with the image processing module; the image acquisition module is used for simultaneously acquiring a first real-time image and a second real-time image of the detected tissue and transmitting the first real-time image and the second real-time image to the image processing module;

the image processing module is connected with the display module; the image processing module is used for determining horizontal parallaxes of the first real-time image and the second real-time image according to a left-right eye parallax relation comparison table and the distance, and adjusting the horizontal parallaxes according to the observation distance, the interpupillary distance, the pupil diameter and the unit pixel width of the display module of an observer; the left-right eye parallax relation table is a table of shooting distance and left-right eye parallax relation.

2. The 3D electronic laparoscope according to claim 1, further comprising a transmission module; the image processing module is respectively connected with the distance sensor and the image acquisition module through the transmission module.

3. The 3D electronic laparoscope according to claim 1, wherein said image processing module is a computer.

4. An image processing method of a 3D electronic laparoscope, applied to the 3D electronic laparoscope according to any one of claims 1-3, comprising:

Acquiring a distance, a first real-time image and a second real-time image; the distance is the distance between the image acquisition module and the detected tissue; the first real-time image and the second real-time image are simultaneously acquired by an image acquisition module;

determining horizontal parallaxes of the first real-time image and the second real-time image according to the distance and the left-right eye parallax relation comparison table;

judging whether the horizontal parallax is within a preset range;

and when the horizontal parallax is not in the preset range, adjusting the distance between the first real-time image and the second real-time image.

5. The image processing method according to claim 4, wherein the preset range is:

Wherein disp _v is the horizontal disparity between the first real-time image and the second real-time image; delta is a constant, D _v represents the distance between the observer and the display, D _e represents the distance between the pupils of both eyes of the human, D represents the pupil diameter size of the human eye, and P _w represents the unit pixel width size of the display.

6. The image processing method according to claim 5, wherein adjusting the distance between the first real-time image and the second real-time image when the horizontal parallax is not within a preset range, specifically comprises:

When (when) When the first real-time image moves away from the second real-time imageThe second real-time image moves away from the first real-time imageA pixel;

When (when) When the first real-time image moves to a direction approaching the second real-time imageThe second real-time image moves/> in a direction approaching the first real-time image

7. The image processing method according to claim 4, wherein the process of obtaining the left-right eye parallax relation table specifically includes:

acquiring a first real-time image sample and a second real-time image sample acquired by an image acquisition module at a plurality of shooting distances;

The first real-time image sample and the second real-time image sample are displayed by the application display module, and left-right eye parallax corresponding to each shooting distance is obtained;

And constructing a left-right eye parallax relation comparison table according to the left-right eye parallaxes corresponding to the shooting distances.

8. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, carries out the steps of the image processing method of a 3D electronic laparoscope as claimed in any one of claims 4-7.

9. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, realizes the steps of the image processing method of a 3D electronic laparoscope as claimed in any one of claims 4-7.