CN107789056B

CN107789056B - A Medical Image Matching Fusion Method

Info

Publication number: CN107789056B
Application number: CN201710976330.8A
Authority: CN
Inventors: 孙品; 刘广伟; 卢云; 张卓立; 于綦悦; 张宪祥
Original assignee: Affiliated Hospital of University of Qingdao
Current assignee: Affiliated Hospital of University of Qingdao
Priority date: 2017-10-19
Filing date: 2017-10-19
Publication date: 2021-04-13
Anticipated expiration: 2037-10-19
Also published as: CN107789056A

Abstract

The invention provides a medical image matching and fusion method, including a simulation process and a matching process, wherein the simulation process is performed before surgery, and ultrasonic image acquisition and storage are performed; Real-time ultrasound images are obtained, and after reaching the lesion area, the matching process is started. The medical image matching and fusion method of the present invention realizes the matching and fusion of the ultrasonic image and the Dicom image during the image-guided operation. The imaging angle of the image is adjusted in real time according to the doctor's operation, which is convenient for the doctor to obtain the intuitive and corresponding ultrasound images and Dicom images, and improves the accuracy of the operation.

Description

Medical image matching and fusing method

Technical Field

The invention relates to the field of medical imaging, in particular to a medical image matching fusion method.

Background

Ultrasonic imaging is to scan a human body by using an ultrasonic sound beam, and obtain an image of an internal organ by receiving and processing a reflected signal. In recent years, ultrasonic imaging techniques such as gray scale display and color display, real-time imaging, ultrasonic holography, transmission ultrasonic imaging, ultrasound parallel tomography, three-dimensional imaging, ultrasonic imaging in a body cavity, and the like have been developed. Ultrasound imaging has been known for its real-time, repeatable, and mobile properties, especially real-time properties, and sonographers use real-time imaging to apply ultrasound in various fields, such as real-time guided puncture, real-time ultrasound contrast imaging, real-time color doppler flow imaging, real-time elastography, and so on.

CT (computed tomography), which is an electronic computer tomography, utilizes precisely collimated X-ray beams, gamma rays, ultrasonic waves and the like to perform section scanning one by one around a certain part of a human body together with a detector with extremely high sensitivity, has the characteristics of short scanning time, clear images and the like, and can be used for checking various diseases; the following can be classified according to the radiation used: x-ray CT (X-CT), ultrasonic CT (uct), and gamma-ray CT (gamma-CT), etc.

MR (magnetic resonance imaging) applies a radio-frequency pulse of a certain specific frequency to a human body in a static magnetic field, so that hydrogen protons in the human body are excited to generate a magnetic resonance phenomenon; after the pulse is stopped, the protons generate MR signals in the relaxation process, and the MR signals are generated through the processes of receiving the MR signals, spatially encoding, reconstructing images and the like. Magnetic resonance imaging is also a type of emission tomography, which uses the phenomenon of magnetic resonance to acquire electromagnetic signals from the body and reconstruct body information.

PET is a novel imaging technology which reflects molecular metabolism and can display biomolecular metabolism, receptor and nerve medium activity on a living body, and the PET is widely used in the aspects of diagnosis and differential diagnosis of various diseases, disease judgment, curative effect evaluation, organ function research, new drug development and the like.

CT/MR/PET imaging is in the Dicom image format, widely used in clinical applications, and accepted with its good resolution and good communication with clinicians, but Dicom image format does not achieve real-time performance, much less mobility.

At present, an image-guided surgery needs an ultrasonic two-dimensional image to position a surgical instrument in real time, but a Dicom image is needed for identifying a focus, because the Dicom image is three-dimensional, the imaging resolution is higher, the visual field is wider, and the focus identification is clearer, so that the current image-guided surgery still needs a doctor to judge and process the focus by taking the ultrasonic image as a guide, the Dicom image obtained before the surgery is referred, a large part of the surgery time is used for image comparison, and the accuracy of the doctor operation is affected by repeated image comparison.

How to provide a method for fusing an ultrasonic image and a Dicom image is a problem to be solved urgently in the field of medical imaging at present.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides a medical image matching and fusing method, which realizes matching and fusing of an ultrasonic image and a Dicom image in an image-guided surgery process, is convenient for doctors to obtain the visual and mutually corresponding ultrasonic image and Dicom image, and improves the accuracy of the surgery.

The technical scheme of the invention is realized as follows:

a medical image matching fusion method comprises a simulation process and a matching process;

wherein, the simulation process is performed before the operation, and comprises the following steps:

step (a1), carrying out ultrasonic scanning on a patient before an operation, carrying out image-guided operation navigation simulation through the ultrasonic scanning, and carrying out ultrasonic image acquisition and storage after reaching a focus area;

step (a2), decomposing the collected ultrasonic image, dividing the ultrasonic image into a plurality of ultrasonic image segments according to the operation process, wherein each ultrasonic image segment corresponds to a group of Dicom images;

step (a3), decomposing each ultrasonic image segment into a frame image sequence, compressing the frame image sequence, extracting a sub-frame image at each fixed interval, compressing the frame image sequence into a group of frame images, marking the focus point in each frame image, marking the front, back, upper and lower parts of the focus point by taking the focus point as the center, and marking the focus area by the marking point;

step (a4), taking each marking point as a center, and carrying out grid division on a focus area, wherein each marking point corresponds to a grid;

step (a5), extracting the texture features of human tissues and the texture features of blood vessels in each grid area to obtain the characteristic values of each marking point in the frame image;

step (a6), collecting the characteristic values of the same mark points in a group of frame images, and collecting the characteristic value sets of the mark points in an ultrasonic image section;

step (a7), storing the characteristic value set of each marking point in the ultrasonic image segment in a subarea manner;

in the formal operation, image-guided operation navigation is carried out through ultrasonic scanning, real-time ultrasonic images are obtained through the ultrasonic scanning, and after the real-time ultrasonic images reach a focus area, a matching process is started;

the matching process comprises the following steps:

step (b1), decomposing the real-time ultrasonic image, dividing the continuous ultrasonic image into frame image sequences, and extracting a frame image at each fixed interval;

step (b2), carrying out grid segmentation on the frame image in the step (b1), wherein the grid size is 1/50-1/20 of the minimum grid in the step (a4), and extracting the texture features of the human tissues and the texture features of blood vessels in each grid after segmentation;

a step (b3) of comparing the human tissue texture features and the blood vessel texture features extracted in the step (b2) with the set of feature values stored in the step (a 7);

a step (b4) of setting the same number and capacity of memory partitions as those in the step (a7), and when the human tissue texture features and the blood vessel texture features extracted in the step (b2) are matched with the feature value set stored in the step (a7), storing the human tissue texture features and the blood vessel texture features of the matched mesh into the corresponding memory partitions;

and (b5) after matching is finished, collecting the human tissue texture features and the blood vessel texture features of each storage partition, matching the real-time ultrasonic image with the ultrasonic image section in the step (a7) when the human tissue texture features and the blood vessel texture feature sets stored in the storage partitions reach 80% or more of the characteristic value set stored in the step (a7), and calling out a corresponding group of Dicom images, wherein the real-time ultrasonic image is matched with the group of Dicom images.

Optionally, in the simulation process and the matching process, decomposing and compressing the continuous ultrasonic image through an external image processing computer; the external image processing computer has the advantages of high operation speed, short fixed interval time, low operation speed and long fixed interval time.

Optionally, in the step (a2), each ultrasound image segment corresponds to a group of Dicom images, the ultrasound image segment is further divided, and the angles of the Dicom images are adjusted according to the positions of the human tissues corresponding to the ultrasound images.

Optionally, in the step (b5), after the matching is finished, the process of collecting the human tissue texture features and the blood vessel texture features of each storage partition includes: and calculating the total capacity of the human tissue texture features and the blood vessel texture features stored in the storage partitions.

Optionally, in the step (b5), when the total capacity of the human tissue texture features and the blood vessel texture features stored in the respective storage partitions reaches 80% or more of the total capacity of the set of feature values stored in the step (a7), the real-time ultrasound image is matched with the ultrasound image segment in the step (a 7).

The invention has the beneficial effects that:

(1) the matching fusion of the ultrasonic image and the Dicom image in the image-guided surgery process is realized;

(2) in the operation process, when the navigation reaches the focus position, the ultrasonic image and the Dicom image which correspond to each other are presented, and the imaging angle of the Dicom image is adjusted in real time according to the operation of a doctor, so that the doctor can conveniently obtain the visual ultrasonic image and the Dicom image which correspond to each other, and the accuracy of the operation is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a schematic view of a method according to the present invention for marking a focal zone;

FIG. 2 is a flow chart of a simulation process of a medical image matching fusion method according to the present invention;

fig. 3 is a flowchart of a matching process of a medical image matching fusion method according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The existing image-guided surgery needs an ultrasonic two-dimensional image to position a surgical instrument in real time, a Dicom image obtained before the surgery is referred to for lesion judgment and treatment, and since the ultrasonic image and the Dicom image do not correspond in real time, a doctor needs a comparison judgment process in the surgical operation process, and the ultrasonic image and the Dicom image are unified, so that time consumption is inevitably caused, and the judgment of the doctor is also hindered.

The invention provides a medical image matching and fusing method, which can effectively fuse a real-time ultrasonic image and a Dicom image of an operation area in the operation process and display the image on a display, so that a doctor can intuitively obtain a real-time navigation image and a focus image, and the operation is convenient.

The medical image matching and fusing method comprises a simulation process and a matching process, wherein the simulation process is performed before an operation, and as shown in fig. 2, the medical image matching and fusing method comprises the following steps:

and (a1) performing ultrasonic scanning on the patient before operation, performing image-guided surgery navigation simulation through the ultrasonic scanning, and acquiring and storing ultrasonic images after reaching a focus area.

In the simulation process, the area needing to be operated in the actual operation process is collected, and images of the same area at different angles are collected according to the actual operation requirement.

The acquisition and storage of the ultrasonic images are realized by an external image processing computer, the external image processing computer can achieve higher speed, and the processing operation of the stored images is also finished in the external image computer, so that the operation space of the computer for image-guided surgery navigation simulation is not occupied.

And (a2) decomposing the acquired ultrasonic image, dividing the ultrasonic image into a plurality of ultrasonic image segments according to the operation process, wherein each ultrasonic image segment corresponds to a group of Dicom images, and the finer the ultrasonic image segment is, the higher the matching degree of the Dicom images and the ultrasonic image segments is.

And (a3) decomposing each ultrasonic image segment into a frame image sequence, compressing the frame image sequence, extracting a frame image at fixed intervals, compressing the frame image sequence into a group of frame images, deleting the frame images which are not extracted, identifying the focus point in each frame image, marking the focus point front, back, upper and lower parts by taking the focus point as the center, and identifying the focus area through the marking point. If the focus points are multiple and continuous, the main axis and the upper, lower, front and back are marked by taking the continuous multiple focus points as the main axis.

Fig. 1 shows a schematic view of the identification of a lesion point, the lesion 10 being marked by a plurality of marking points 20.

And (a4) meshing the lesion area with each marker as the center, wherein each marker corresponds to a mesh. Because the distance between the mark points is not fixed, the size and the shape of the grid divided by taking each mark point as the center are not fixed, and the grid division principle is as follows: and dividing grids by taking the vertical central lines of two adjacent marking points as boundary lines, wherein the marking points are positioned at the edge positions, and the grid lines at the outer sides are arranged according to the grid lines at the inner sides.

And (a5) extracting the human tissue texture features and the blood vessel texture features in each grid area to obtain the feature values of each marking point in the frame image.

Because the human tissue texture and the blood vessel texture have higher identification degree, and the coincidence probability of the human tissue texture or the blood vessel texture in different areas is lower than one millionth, the human tissue texture feature and the blood vessel texture feature are simultaneously extracted, and the coincidence probability of the texture features is further reduced. Each mesh region can be uniquely identified by human tissue texture features and blood vessel texture features.

And (a6) collecting the characteristic values of the same mark points in a group of frame images and collecting the characteristic value sets of the mark points in the ultrasonic image section.

Since the ultrasound image segment is decomposed and compressed into a group of frame images, the feature value of a mark point in the group of frame images is collected to obtain the feature value set of the mark point in the ultrasound image segment.

And similarly, collecting the characteristic values of all the mark points in a group of frame images, and collecting the characteristic values of all the mark points in the ultrasonic image section.

And (a7) storing the characteristic value set of each marking point in the ultrasonic image segment in a subarea manner, wherein each storage subarea stores the characteristic value set of one characteristic point. For example, a group of compressed frame images has 100 frames of pictures, one marker has 100 feature values, the 100 feature values of the marker are stored in one storage partition, the 100 feature value sets stored in the storage partition correspond to the feature value of the marker in the ultrasound image segment, and each marker in the ultrasound image segment has a separate storage partition for storing the feature value sets.

Since the simulation process matches the acquired ultrasound image segment with the Dicom image, in the formal surgery, if the real-time ultrasound image at a certain moment matches the acquired ultrasound image segment, the real-time ultrasound image at the moment matches with the corresponding Dicom image group. Next, the matching process of the ultrasound image segment and the Dicom image will be described in detail.

In the formal operation, image-guided surgical navigation is performed through ultrasonic scanning, the ultrasonic scanning obtains a real-time ultrasonic image, and after the real-time ultrasonic image reaches a focus area, a matching process is started, as shown in fig. 3, the matching process comprises the following steps:

and (b1) decomposing the real-time ultrasonic image, dividing the continuous ultrasonic image into a frame image sequence, and extracting one frame image at each fixed interval.

The method is characterized in that the method comprises the steps of decomposing a real-time ultrasonic image through an external image processing computer, wherein the external image processing computer is higher in speed, the external image processing computer synchronously decomposes the ultrasonic image in the real-time ultrasonic image generating and displaying process, continuous ultrasonic images are divided into frame image sequences, and one frame image is extracted at each fixed interval.

And (b2) carrying out grid segmentation on the frame image in the step (b1), wherein the grid size is 1/50-1/20 of the minimum grid in the step (a4), and extracting the texture features of the human tissues and the texture features of the blood vessels in each grid after segmentation.

In the step (a4), the size and the shape of the grid are not fixed, so that the minimum grid is selected, the frame image obtained in the step (b1) is subjected to grid division by taking 1/50-1/20 of the grid as a standard, and the texture features of the human tissue and the texture features of the blood vessels in each divided grid are extracted.

Step (b3), comparing the human tissue texture features and the blood vessel texture features extracted in step (b2) with the feature value set stored in step (a7), wherein the comparison process specifically comprises the following steps:

comparing the texture features of the human tissues and the texture features of the blood vessels of the meshes to be compared with the characteristic values in the storage subareas in the step (a7), wherein the meshes to be compared are smaller, so that the meshes to be compared only generate certain matching errors at the edge positions of the lesion areas and are completely matched with the inner areas of the lesions.

In the step (b5), after the matching is finished, the process of collecting the human tissue texture features and the blood vessel texture features of each storage partition is specifically as follows: and (3) calculating the total capacity of the storage space occupied by the human tissue texture features and the blood vessel texture features stored in each storage partition, wherein the stored human tissue texture features and the blood vessel texture features are both matched grids, so that the total capacity of the storage space occupied by the matched grids is calculated and compared with the total capacity of the storage space of the characteristic value set stored in the step (a 7). When the total capacity of the human tissue texture features and the blood vessel texture features stored in the storage partitions reaches 80% or more of the total capacity of the feature value set stored in the step (a7), the real-time ultrasound image is matched with the ultrasound image segment in the step (a 7).

By the matching fusion method, the ultrasonic images are displayed on the display at each stage in the operation, the Dicom images corresponding to the ultrasonic images are synchronously displayed, and doctors synchronously refer to the corresponding Dicom images while observing the ultrasonic images, so that the operation efficiency is improved.

In the simulation process and the matching process, decomposing and compressing the continuous ultrasonic images through an external image processing computer; the external image processing computer has the advantages of high operation speed, short fixed interval time, low operation speed and long fixed interval time.

In the operation process, the Dicom image with a fixed angle cannot meet the operation requirement of a doctor, because the real-time ultrasonic image needs to adjust the angle according to the operation instrument according to the operation requirement, although the operation area is unchanged, the operation angle is changed, and the real-time ultrasonic image and the Dicom image have certain angle deviation.

The invention provides a medical image matching and fusing method, which realizes matching and fusing of an ultrasonic image and a Dicom image in an image-guided operation process, and in the operation process, when a focus position is reached, the ultrasonic image and the Dicom image which correspond to each other are displayed on a display, and the imaging angle of the Dicom image is adjusted in real time according to the operation of a doctor, so that the doctor can conveniently obtain the visual ultrasonic image and the Dicom image which correspond to each other, and the accuracy of the operation is improved.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A medical image matching and fusion system, comprising an external image processing computer and a display for performing the following steps: comprising a simulation process and a matching process;

Among them, the simulation process is carried out before surgery, including the following steps:

In step (a1), the collected ultrasonic images are decomposed, and divided into multiple ultrasonic image segments according to the operation process, and each ultrasonic image segment corresponds to a group of Dicom images;

Step (a2), decompose each ultrasound image segment into a frame image sequence, compress the frame image sequence, extract a pair of frame images at a fixed interval, compress the frame image sequence into a set of frame images, and analyze the lesions in each frame image. Mark the point, take the lesion point as the center, mark its front and back, up and down, and mark the lesion area through the mark point;

Step (a3), with each marked point as the center, the lesion area is divided into grids, and each marked point corresponds to a grid;

Step (a4), extracts the texture feature of human body tissue and the texture feature of blood vessels in each grid area, and obtains the feature value of each marked point in the frame image;

Step (a5), assemble the eigenvalues of the same marked point in a group of frame images, into the eigenvalue set of this marked point in the ultrasound image segment;

Step (a6), the feature value set of each marked point in the ultrasound image segment is stored by partition;

The matching process includes the following steps:

Step (b1), decompose the real-time ultrasound image, divide the continuous ultrasound image into a frame image sequence, and extract a pair of frame images every fixed interval;

In step (b2), the frame image in step (b1) is divided into grids, and the grid size is 1/50~1/20 of the smallest grid in step (a4), and the human tissue in each grid after segmentation is extracted. Texture features and blood vessel texture features;

Step (b3), comparing the human tissue texture feature and blood vessel texture feature extracted in step (b2) with the feature value set stored in step (a6);

Step (b4), set the same number and capacity of storage partitions as in step (a6), when the texture features of human tissue and blood vessels extracted in step (b2) match the set of eigenvalues stored in step (a6) , and store the human tissue texture features and blood vessel texture features of the matched meshes into the corresponding storage partitions;

Step (b5), after the matching, the human tissue texture features and blood vessel texture features of each storage partition are collected, and when the human tissue texture features and blood vessel texture feature sets stored in the storage partition reach the feature value stored in step (a6) 80% or more of the set, the real-time ultrasound image matches the ultrasound image segment in step (a6), and a corresponding group of Dicom images is called out, and the real-time ultrasound image matches the group of Dicom images.

2. The medical image matching and fusion system according to claim 1, wherein in the step (a1), each ultrasound image segment corresponds to a group of Dicom images, and the ultrasound image segment is further divided, according to the ultrasound image segment. Adjust the angle of the Dicom image according to the position of the human tissue corresponding to the image.

3. A kind of medical image matching and fusion system as claimed in claim 1, it is characterized in that, in described step (b5), after matching, the process that the human tissue texture features and blood vessel texture features of each storage partition are collected , specifically: calculating the total capacity of the storage space occupied by the human tissue texture features and blood vessel texture features stored in each storage partition.

4. a kind of medical image matching and fusion system as claimed in claim 3, is characterized in that, in described step (b5), when the human tissue texture feature and the blood vessel texture feature stored in each storage subarea account for the total amount of storage space occupied. When the capacity reaches 80% or more of the total capacity of the feature value set stored in step (a6), the real-time ultrasound image matches the ultrasound image segment in step (a6).