The invention content is as follows:
in view of the above problems in the prior art, the present invention is intended to provide an operation training method for microwave (or radio frequency, etc.) ablation operation based on real objects, which can realize direct video feedback. In the image processing, the invention adopts the means of coaxial shooting, overflow correction, heat conductivity correction and the like, thereby ensuring the maximum approximation of the simulated environment and the real environment.
In one aspect, the present invention provides a method for simulating an interventional operation based on video image feedback (or may also be referred to as a method for constructing an operation simulation environment), where the method includes:
(1) Constructing a human body simulation model, wherein the human body simulation model at least comprises a simulation tissue layer and a simulation organ, and at least the simulation organ is made of a transparent material;
(2) arranging a video monitoring device for an ablation process at one side of the simulated visceral organ, wherein the video monitoring device has a video image acquisition function of at least two wave bands;
(3) during a surgical simulation with the ablation device, the video monitoring device is notified to perform a video acquisition on at least one of the two acquisition bands by an ablation switch of the ablation device.
Preferably, a first band of the two bands is a visible light band or a shorter wavelength light band, and the second band is an infrared band.
Preferably, the simulation process includes a puncturing stage and an ablation stage, an ablation switch of the ablation device is in communication connection with the video monitoring device for mode switching, the puncturing stage adopts a first wave band for video acquisition, the ablation stage adopts a second wave band for video acquisition or acquires a fusion image of the first and second wave bands, and the fusion image is presented to a user.
Preferably, the fused image is formed as follows:
(3.1) extracting RGB values of the visible light image, and weighting the RGB values respectively, wherein the weighting coefficient of each value is less than or equal to 45% or 50%;
(3.2) acquiring an infrared image by using infrared camera equipment, and weighting the RGB value of the infrared image, wherein the weighting coefficient is more than or equal to 45% or 50%;
and (3.3) performing pixel-by-pixel fusion on each frame of the infrared image and the visible light image according to the synchronous signals, and adding the RGB values of the two images respectively.
Preferably, R of each pixel point in the fused image is fused
Melt、G
Melt、B
MeltComparing the values with corresponding overflow thresholds respectively, determining the proportion of overflow pixels in the total pixel value, and multiplying the pixel value of any one of RGB by an anti-overflow coefficient when the proportion of the overflow pixels in the total pixel exceeds a preset threshold, wherein the anti-overflow coefficient is as follows:

where Q is the overflow threshold, A is the average pixel value, and Y is the overflow ratio. The coefficient is adopted to reduce the pixel value of the corresponding item, the higher the overflow proportion is, the larger the reduction amplitude is, and the overflow phenomenon is effectively avoided. Then, iterating and calculating the overflow proportion again, if the overflow phenomenon still occurs, reducing the coefficient according to the range of 0.05-0.1 (namely 5% -10%) each time, if the overflow proportion still exceeds the limit after three times of reduction, judging the picture to be abnormal, abandoning the picture, adopting the array value of the equal difference number of the corresponding pixels of the first three frames of the picture as the pixel value of the pixel, for example, setting the pixel value range as [0,255 ] ]For example, if a pixel value is abnormal, the G pixel values of the pixels in the first three frames are 172, 176 and 184 respectively, and the distance between every two pixels is calculatedThe average value of the difference, 7, is set to 191. Therefore, the image change trend can be kept, and excessive distortion is avoided.
Preferably, the method further comprises: and establishing a corresponding relation mapping of pixels of the ablation monitoring equipment and the temperature, and highlighting or marking the specific temperature value concerned by a doctor in the ablation process based on the corresponding relation mapping.
Preferably, the method further comprises: measuring the heat conductivity coefficient of the simulated organ material and the heat conductivity coefficient of the animal liver material, firstly weighting the infrared image based on the material, and then fusing, wherein the weighting mode is as follows:
p' is the pixel value after correction, P is the pixel value before correction, λ
1And λ
2Thermal conductivity of the measured animal tissue and the thermal conductivity of the simulation material, c
1And c
2Specific heat of tested animal tissue and specific heat of simulated material, rho
1And ρ
2The density of the tested animal tissue (such as liver) and the density of the simulated material, respectively.
Preferably, the method further comprises repairing the punctured simulated tissue layer by a repair patch, and removing and replacing the simulated organ by detaching the detached upper body part model and lower body part model.
Preferably, the video monitoring device adopts a coaxial light beam splitting collection mode to collect images of different wave bands.
In another aspect, the present invention provides an operation training system for minimally invasive ablation procedures, the operation training system comprising: human simulation model, ablation equipment, operation panel, ablation monitoring facilities and ablation needle state display device, human simulation model set up on the operation panel, and human simulation model includes the upper part of the body model at least, the upper part of the body model is close to viscera one side and sets up ablation monitoring facilities, and the upper part of the body model is inside to have the simulation internal organs, the simulation internal organs is made by transparent material, it includes visible light imaging device and infrared thermal imaging device to melt monitoring facilities, and visible light imaging device and infrared thermal imaging device all are to the target internal organs, camera equipment and thermal imager with it melts needle state display device and is used for the image based on the control signal transport different grade type of ablation equipment.
By adopting the operation training simulation method, the user can repeatedly practice the operation, and the corresponding relation between the hand operation action and the advancing distance of the puncture needle can be known by the user through the video-based insertion depth feedback, so that the user can conveniently control the action amplitude of the user, know the instrument action condition caused by the action amplitude, and provide better control feeling. And the visual image and the infrared image of the target organ are switched and displayed or the visual image and the fusion image are switched and displayed by triggering the ablation switch, so that the visual display of the ablation time and the temperature change of the ablation area can be realized, and a user can better know the influence of the operation on the target tumor. The existing operation mode is usually to distinguish the ablation state by changing color, but the color change cannot accurately reflect the ablation temperature, and the temperature change caused by ablation can be effectively and really reflected by heat conductivity correction, temperature measurement of a temperature sensor and mapping.
The specific implementation mode is as follows:
as shown in fig. 1, the method for simulating an interventional operation based on video image feedback of the present invention comprises:
(1) constructing a human body simulation model, wherein the human body simulation model at least comprises a simulation tissue layer and a simulation organ, and at least the simulation organ is made of a transparent material;
(2) arranging a video monitoring device for an ablation process at one side of the simulated visceral organ, wherein the video monitoring device has a video image acquisition function of at least two wave bands;
(3) during a surgical simulation with the ablation device, the video monitoring device is controlled by an ablation switch of the ablation device to perform video acquisition on at least one of two acquisition bands. As shown, before ablation starts, i.e. during the puncture phase, image acquisition is performed with visible light, and after ablation starts, image acquisition and display is performed with infrared light or display is performed with a fusion image.
The fused image is formed as follows:
(3.1) extracting RGB values of the visible light image, and weighting the RGB values respectively, wherein the weighting coefficient of each value is less than or equal to 45% or 50%;
(3.2) acquiring an infrared image by using infrared camera equipment, and weighting the RGB value of the infrared image, wherein the weighting coefficient is more than or equal to 45% or 50%;
and (3.3) performing pixel-by-pixel fusion on each frame of the infrared image and the visible light image according to the synchronous signals, and adding the RGB values of the two images respectively.
As shown in fig. 2, the system used in the training method for minimally invasive ablation operation in this embodiment includes: the manikin 100, the ablation device 200, the console 300, the ablation monitoring device 400, and the ablation needle motion display device 500.
The operating table 300 is constructed substantially the same as or similar to an operating table in order to give the trainee as a realistic experience as possible.
For example, for liver cancer treatment training, the phantom 100 may be divided into two parts, one part being an upper part having visceral tissues, the other part being a lower part (which may be omitted), and the other part being a more realistic phantom having better quality, the other part being a lower part (which may be omitted), the other part being a common phantom having lower cost, thereby further reducing the cost. And the upper body part and the lower body part can be assembled together in two parts so as to be disassembled and assembled, and when the upper body part needs to be replaced, the lower body part does not need to be replaced at the same time. More preferably, a human head model (which can be omitted) is further included, and the human head model and the upper body part model are detachably assembled together.
A hollow region is provided below the phantom 100 (the lower portion of the upper phantom), and the hollow region is located in a non-visceral region (particularly, a non-liver region) near the liver portion. The purpose of the hollow region is to position ablation monitoring device 400, with the height of ablation monitoring device 400 being proportional to the height of the target monitoring region. At least the inner part (particularly, the internal organs) of the upper body model of the manikin 100 is made of a transparent material, such as transparent silica gel.
The ablation monitoring device 400 includes a visible light imaging device 401 and/or an infrared thermal imaging device 402, both of which are directed at the target organ. The target organ is made of a silica gel model or other transparent or semitransparent materials, more preferably, the heat capacity and the heat conductivity of the target organ are made of materials similar to the heat conductivity and the heat capacity of human tissues, and simulated tumors in various shapes and structures can be plugged into the target organ according to requirements.
The imaging focal points of the visible light imaging device 401 and the infrared thermal imaging device 402 are directed to a target simulated lesion region of a target organ.
In a preferred implementation, the visible light imaging device 401 and the infrared thermal imaging device 402 are arranged side by side, and the two images respectively capture images of the target organ and the simulated tumor from respective angles and respective wave bands. However, there is a problem that the user needs to observe the state of the two images at the same time, and since the two images are photographed side by side, the photographing angles of the two images are not completely the same but are shifted by a certain distance in the horizontal or vertical direction, so that the contents of the photographed images are not directed to the same portion. This easily imposes a burden of observation on the user.
To address this problem, in another preferred implementation, an optical path structure as shown in fig. 3 is adopted to perform coaxial and spectroscopic image acquisition. As shown, the ablation monitoring device 400 includes a visible light imaging device 401, an infrared thermal imaging device 402, an optical lens 403, a beam splitter 404, a reflector 405, and a housing 406, where the optical lens 403 is located at the frontmost end of the housing, the housing has an opening, the optical lens 403 is just sealed at the opening, and the beam splitter 404 is located behind the optical lens 403, and is used for receiving incident light from the optical lens 403 and performing sub-band reflection and transmission on the incident light, where the beam splitter 404 is plated with a reflective film for visible light or infrared light, so as to reflect one of the visible light or infrared light and transmit light of another band, for example, the visible light is transmitted to the visible light imaging device 401, the infrared light is reflected to the reflector 405, and then reflected to the infrared light imaging device 402.
In this way, the same field of view of the visible light imaging device 401 and the infrared light imaging device 402 can be achieved.
Further, in order to make the observation of the user more convenient, the specific operation flow of the surgical simulation training method may be divided into two modes, a puncturing mode and an ablation mode, wherein in the puncturing mode, the image obtained by the visible light imaging device 401 is presented to the user through the display device 500, so that the user can know the corresponding relationship between the hand operation action and the travel distance of the puncturing needle, the user can control the action amplitude of the user conveniently, know the instrument action condition brought by the action amplitude, and provide better control feeling.
When the puncture reaches a preset position, an ablation mode is started, a user triggers a control switch of an ablation needle, the control switch simultaneously sends a mode switching signal to the ablation monitoring device, and the ablation monitoring device fuses an image obtained by the visible light imaging device 401 and an image obtained by the infrared light imaging device 402.
The specific fusion mode is that shutter triggering is performed on synchronous trigger signals of the visible light imaging device 401 and the infrared light imaging device 402, and visible light images and infrared light images of corresponding frames are acquired based on the synchronous signals. RGB values of a visible light image are extracted, and the RGB values are weighted unequally, and in the weighting, a weight w with an R value smaller than G, B is givenR1、wG1、wB1Preferably, the weight assigned to the R value is less than about 10-20% of the weight of the G, B value. The respective weighting coefficients of RGB may be set to between 30% and 60%, preferably 50% or less. The RGB values of the infrared photographic image obtained by the infrared light imaging device 402 are extracted, and since infrared light is invisible to the human eye, the infrared light imaging device 402 converts the infrared light image captured by it into an image visible to the human eye, where the object of the operation performed therein is the converted infrared image. The RGB values of the infrared image are weighted unequally, wherein the weighting coefficient for the R value is larger than the weighting coefficient w of GB R2、wG2、wB2. By adopting the weighting mode, on one hand, the tone of the visible light image can be adjusted towards the short wave direction, and the tone of the image formed by the infrared light imaging equipment can be adjusted towards the long wave direction, so that once the infrared light image enters the ablation stage, the heat conduction condition of the ablated area can be reflected more clearly, and on the other hand, the possibility of overflow of the pixel values of the weighted and fused two images can be reduced. And controlling the two image acquisition devices to output images with the same resolution, so that better fusion can be realized, and certainly, before the fusion, the registration of the two images can be carried out by adopting a conventional image registration method.
R of the weighted visible light image1The value and the corresponding pixel R of the weighted infrared image2Adding the values, and weighting the G of the visible light image1Value and G of weighted infrared image2Adding the values, and weighting B of the visible light image1Value and B of weighted infrared image2Adding values to obtain RGB values of pixels corresponding to the fused image: rMelt、GMelt、BMelt。
Preferably, the method further comprises fusing R of each pixel point in the image
Melt、G
Melt、B
MeltComparing the values with corresponding overflow thresholds respectively, determining the proportion of overflow pixels to the total pixel value, taking possible noise spikes into account, multiplying the pixel value of any one of RGB by an anti-overflow coefficient when the proportion of overflow pixels to the total pixel exceeds a predetermined threshold, and multiplying the anti-overflow coefficient by the pixel value of the corresponding one of RGB May be set based on the average pixel value of the term, the overflow threshold, and the overflow ratio, for example, the coefficient is set to:

wherein Q is an overflow threshold, A is an average pixel value, and Y is an overflow proportion, and the coefficient is adopted to reduce the pixel value of the corresponding item so as to avoid the overflow phenomenon. Then, iterating and calculating the overflow proportion again, if the overflow phenomenon still occurs, reducing the coefficient according to the range of 0.05-0.1 (namely 5% -10%) each time, if the overflow proportion still exceeds the limit after three times of reduction, judging the picture to be abnormal, abandoning the picture, adopting the array value of the equal difference number of the corresponding pixels of the first three frames of the picture as the pixel value of the pixel, for example, setting the pixel value range as [0,255 ]]For example, if a pixel value is abnormal, the G pixel values of the pixels in the first three frames are 172, 176, 184 respectively, the average value of the difference values between the two is calculated, 7, and the current pixel value is set to 191. Therefore, the image change trend can be kept, and excessive distortion is avoided.
Further, a corresponding relation mapping of pixels of the ablation monitoring device and temperature is established, a temperature sensor is arranged at a preset position (distance) of the simulated organ, the ablation electrode of the ablation needle is inserted into the simulated organ, the distance of 1-4cm is kept between the ablation electrode and the temperature sensor, the power of the ablation electrode is gradually increased, measurement trigger signals are respectively sent to the temperature sensor and the ablation monitoring device, the temperature sensor measures the actual temperature of the position point where the temperature sensor is located, and the ablation monitoring device obtains a fusion image of the ablation electrode and the temperature sensor. Extracting the pixel value of the simulated organ close to the temperature sensor through image extraction software, establishing the corresponding relation between the pixel value and the temperature value, repeatedly testing to obtain the temperature value corresponding to each pixel value section, and forming a mapping relation table between the pixel value and the temperature value under the current parameter condition. Based on the relation table, a mark or a pixel expansion display is performed for a specific temperature value (specific pixel value) concerned by a doctor in the ablation process. For example, for a pixel value corresponding to a specific temperature, such as 120 degrees celsius, an isotherm is formed, and a pixel and its surrounding pixels at the isotherm are assigned with a higher brightness value, or pixels 3 × 3 around the pixel at the isotherm are assigned with the same RGB values, so as to form a distinct marking region. According to different tumors or experimental conditions, a plurality of parameter systems and corresponding mapping relations can be established.
Further, the thermal conductivity, specific heat and density of the material of the simulated organ (tumor) are measured, the thermal conductivity, specific heat and density of the material of the animal liver (tumor) are measured, and the pixel values are weighted based on the material by the following weighting method:
p' is the pixel value after correction, P is the pixel value before correction, λ
1And λ
2Thermal conductivity of the measured animal tissue and the thermal conductivity of the simulation material, c
1And c
2Specific heat of tested animal tissue and specific heat of simulated material, rho
1And ρ
2The density of the tested animal tissue (such as liver) and the density of the simulated material, respectively.
More preferably, in order to reduce the cost better, a replaceable organ model and/or an upper body surface tissue model are provided, and the organ model and the upper body surface tissue model are made of double-component silica gel. Preferably, the upper body model comprises tissue layers and a skeleton (the skeleton is used to form skeleton contours, and the tissue layers, especially the outer tissue layers, are used to cover the skeleton, and for the puncture region we are concerned with, there are no skeleton layers, as it is necessary to puncture from the skeleton space or the abdominal frameless region). The tissue layers comprise an outer tissue layer 3 and an inner substrate layer 4, the outer tissue layer and the inner substrate layer are attached to each other, the two ends are hermetically connected, the two are fixed at intervals and are approximately attached to each other, and a fluid channel is formed between the two.
The inner substrate is attached to the inner side of the outer tissue layer, and a plurality of attaching fixing points are arranged at intervals between the inner substrate and the outer tissue layer in a bonding or other fixing mode, so that the inner substrate and the simulated tissue layer are fixed together, and flexible glue is used, is dispensed and is bonded together. The periphery of the inner substrate is hermetically connected with the inner wall of the simulated tissue layer, a filling layer is formed between the inner substrate and the simulated tissue layer, a filling liquid input port 8 and a filling liquid suction port 9 are respectively arranged at the joint part of the inner substrate and the simulated tissue layer and at the two ends of the joint part, and the filling liquid input port and the filling liquid suction port are respectively communicated with the filling layer between the inner substrate and the inner simulated tissue layer. Preferably, the inner substrate is made of a material with little viscosity, such as flexible silicone, to ensure a tight fit between the inner substrate and the simulated tissue layer. Preferably, the backing layer is provided only on one side of the simulated tissue layer adjacent to the anterior chest and not on the other side, since most surgical ablation procedures are performed with a puncture from the anterior side.
More preferably, the system further includes a repair patch 501, the repair patch includes a circular or square patch, one side of the patch has a smooth surface, at least the peripheral part of the other side surface is a sticky surface, the middle part of the sticky surface is movably bonded with a circular repair block 502, the circular repair block is attached with a first component of the two-component silica gel (in this embodiment, the repair block includes a cover film and a first component of the silica gel therein, the cover film can be broken by squeezing under external force), and the first component silica gel is matched with a second component silica gel input in the filling liquid input port to form the two-component silica gel. Preferably, the first component of the circular repair block is a silica gel curing agent, or the silica gel curing agent is filled in the circular repair block, and the silica gel curing agent is matched with silica gel liquid input in the filling liquid input port and is two components in two-component silica gel respectively. More preferably, the two-component silica gel is colorless or pure color, flexible gel.
Preferably, the two-component silica gel consists of a first component and a second component, wherein the mass ratio of the second component to the first component is 10: 1. the first component comprises 2-3 parts of a cross-linking agent, 1-4 parts of a coupling agent, 0.1-0.3 part of a catalyst, 5 parts of a plasticizer and 3 parts of a chain extender, the second component comprises 100-120 parts of hydroxyl-terminated polysiloxane, 80 parts of a filler and 30 parts of a plasticizer, the filler can be heavy calcium carbonate, silicon micropowder and aluminum hydroxide, the plasticizer is methyl silicone oil, and the viscosity of the hydroxyl-terminated polysiloxane is 400-650 mPa. The cross-linking agent is at least one of methyl orthosilicate or ethyl silicate, methyl triethoxysilane and tetraisopropoxy silane, the coupling agent is vinyl trimethoxysilane or aminopropyl trimethoxysilane, the chain extender can be a conventional common chain extender, such as dimethyl diethylsilane, and the aminopropyl methyl dimethoxysilane catalyst is dibutyltin dilaurate.
The repairing process comprises the following steps: and respectively pasting a repairing patch 101 on each of two sides of each puncture hole, namely the inner side of the inner substrate layer and the outer side of the simulated tissue layer, wherein the middle part of each repairing patch is provided with a circular repairing block or is coated with a first component of repairing liquid, and the repairing patches are opposite to the puncture holes. Then, the filling liquid input port is opened, and the filling liquid suction port is naturally opened to discharge residual air possibly existing in the filling layer. The second component silica gel is slowly filled into the filling layer through the filling liquid inlet, and if the filling is not uniform, the second component silica gel can be manually pressed and adjusted. Because the inner substrate layer and the simulated tissue layer are originally attached together and fixedly connected at intervals, a fluid channel is formed between the inner substrate layer and the simulated tissue layer after the second component adhesive is filled, then the second component is conveyed to each position between the two components, including each puncture position, the air remained in the original puncture is extruded to the other side of the filling layer, so that the filling liquid is filled into the puncture and is contacted with the repair block on the repair patch or the first component of the repair liquid, after the second component is filled in each puncture hole, the filling liquid input port is closed, the filling liquid suction port is opened, the filling liquid is slowly sucked outwards until the inner substrate layer and the simulated tissue layer are basically attached together again, because the filling liquid input port and the filling liquid suction port are respectively positioned at two sides of the simulation equipment, residual air in the original puncture hole can be pumped out in the suction process, and the second component adhesive is left in the pit of the puncture hole. At the moment, the repairing blocks on the puncture holes can be gently squeezed and kneaded, so that the first component adhesive and the second component on the repairing blocks are fully mixed, then the repairing blocks are kept still until the curing is completed, the repairing of the inner substrate layer and the simulated tissue layer is realized, and the residual second component in the repairing blocks can be further sucked. After curing, the repair will become a further fixation point between the two layers. By adopting the liquid curing repairing mode, the repairing position has almost no obvious boundary, and the repairing position is better fused with the original material. Most preferably, the inner backing layer and the simulated tissue layer are made of the same material as the repair material, i.e., they are also made of two-component silicone or other similar two-component curing material. In another preferred implementation, the repair patch is separable from the inner simulated tissue layer and the repair patch is torn off when the repair is complete.
While the principles of the invention have been described in detail in connection with the preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing embodiments are merely illustrative of exemplary implementations of the invention and are not limiting of the scope of the invention.