JP7430249B2 - Image processing device, image display system, image processing method and program - Google Patents

Description

The present invention relates to an image processing device, an image display system, an image processing method, and a program.

Techniques for automatically detecting objects from images have been widely studied. For example, in the medical field, technology is being developed to automatically detect lesions such as pulmonary nodules, fractures, subarachnoid hemorrhages, and cerebral infarctions from CT images. Generally, superimposed display of bounding boxes, mask images, etc. is applied to display the results of automatic detection. Note that CT is an abbreviation for Computed Tomography.

Patent Document 1 describes a medical image processing system that analyzes a medical image, generates diagnostic support information based on the analysis result, and displays the diagnostic support information using a display device. The system described in this document generates a diagnostic support image suitable for referring to diagnostic support information.

The diagnostic support images applied to this system are superimposed with scales and equally spaced grid patterns. This makes it easier to recognize the position, size, etc. of the structure in the diagnosis support image.

Patent Document 2 describes a medical image capturing system that generates a still image of the subject by irradiating a specific region of a patient with radiation. The system described in this document performs reduction processing on a medical image and generates a preview image. The system divides the irradiation field of the preview image into multiple small regions and extracts features from the signal values of each small region.

Patent Document 3 describes an ultrasonic diagnostic apparatus that generates an ultrasonic image of a subject. The apparatus described in this document superimposes a color Doppler image of a region of interest on a B-mode tomographic image and displays it using a display unit. The device determines whether the pixel is included in the region of interest based on the pixel value of the designated pixel.

Patent Document 4 describes an image display device that reads a CT image captured using an image capturing device, generates various images for medical diagnosis, and displays the generated images using a screen. This document discloses voxel data obtained from CT images.

Japanese Patent Application Publication No. 2005-103055 Japanese Patent Application Publication No. 2013-102851 Japanese Patent Application Publication No. 2007-190172 Japanese Patent Application Publication No. 2008-100107

However, when the bounding boxes are displayed in a superimposed manner, it is difficult to tell where the detection results are. Bounding boxes can be displayed in isolated areas, but if multiple bounding boxes are displayed on one screen, the screen may become difficult to see.

Regarding the superimposed display of mask images, the extent to which the lesion area is determined may differ from doctor to doctor. When a doctor's judgment and a CAD (computer-aided diagnosis) judgment differ, the reliability of the CAD may become a problem.

The original purpose of CAD is to prevent lesions from being overlooked. On the other hand, CAD shows the area of the lesion to the doctor. Even if CAD is put on the market for the purpose of preventing missed lesions, there is a possibility that CAD will be used in a way that exceeds the goal of preventing missed lesions, resulting in a biased view of diagnosis.

The grid pattern described in Patent Document 1 is used when recognizing the position and size of a structure, and depending on the aspect of the grid pattern, the visibility of the CAD detection result can be reduced.

The grid illustrated in FIG. 5A of Patent Document 2 is a small area obtained by dividing the preview image into a 10×10 matrix. The small area is a unit for calculating feature amounts, and is not related to the display of CAD detection results.

The grid illustrated in FIG. 4 of Patent Document 3 is a unit of processing when extracting a region of interest, and is not related to the display of CAD detection results.

Patent Document 4 does not include any description or suggestion regarding the display of CAD detection results.

The present invention has been made in view of the above circumstances, and provides an image processing device, an image display system, an image processing method, and a program that can display detection results without impairing the reliability of detecting feature regions. The purpose is to

In order to achieve the above object, the following invention aspects are provided.

An image processing device according to the present disclosure is an image processing device including one or more processors, the processor acquires a medical image obtained by photographing a subject, detects a characteristic region from the medical image, A display unit grid having a size that is an integer multiple of 2 or more of the pixels constituting the medical image, and a display unit grid having a size that exceeds the processing unit size for detecting a feature region is called a medical image. , and generates a display signal that causes a display frame having one or more display unit grids corresponding to the size of the feature region to be displayed superimposed on the feature region at a position corresponding to the position of the feature region. .

According to the image processing device according to the present disclosure, the feature region detected from the medical image has a size that is an integral multiple of 2 or more of the pixels constituting the medical image, and the size of the processing unit of the medical image By overlapping display frames having one or more display unit grids having a size exceeding , it is possible to display the detection results without impairing the reliability of detecting the feature region.

Feature regions may include pulmonary nodules, fractures, hemorrhages, cerebral infarctions, and the like. The feature region may include a lesion. Multiple characteristic regions can be detected from one medical image.

In an image processing device according to another aspect, the processor generates a display signal that displays the outline of the display frame using a manner different from the outline of the display unit grid.

According to this aspect, it is possible to make the display frame stand out on the screen where the medical image is displayed.

In an image processing device according to another aspect, the processor generates a display signal that causes the outline of the display frame to be displayed outside the outline of the display unit grid serving as the edge of the feature region.

According to this aspect, it is possible to make the display frame stand out with respect to the display grid.

In an image processing device according to another aspect, the processor generates a display signal that hides the display unit grid.

According to this aspect, the display frame can be made to stand out.

In an image processing device according to another aspect, the processor generates a mask image corresponding to the feature region and generates a display signal representing the mask image.

According to this aspect, a mask image is also used for the display frame. This can improve the visibility of the feature area.

In an image processing device according to another aspect, the processor sets the size of the display unit grid according to the type of feature region.

According to this aspect, a display frame can be displayed according to the type of feature region.

In an image processing device according to another aspect, the processor uses the center of gravity of the medical image and the center of gravity of the display grid to align the positions of the medical image and the display grid.

According to this aspect, the center of gravity of the medical image, the center of gravity of the display grid, the medical image to be used, and the display grid can be aligned.

In an image processing device according to another aspect, the processor aligns the medical image and the display grid using the center of gravity of the subject and the center of gravity of the display grid.

According to this aspect, it is possible to align the center of gravity of the subject, the center of gravity of the display grid, the medical image to be used, and the display grid.

In an image processing device according to another aspect, the processor sets a display grid in which two-dimensional display unit grids are arranged two-dimensionally.

According to this aspect, visibility of characteristic regions can be improved in a two-dimensional medical image such as a tomographic image.

In an image processing device according to another aspect, the processor sets a display grid in which three-dimensional display unit grids are arranged three-dimensionally.

According to this aspect, the visibility of the characteristic region can be improved in a three-dimensional medical image.

In an image processing device according to another aspect, the processor detects the characteristic region as a polygon.

According to this aspect, display frames can be displayed in a superimposed manner on a polygonal feature region.

An image display system according to the present disclosure includes an image processing device including one or more processors, and a display that receives a display image signal transmitted from the image processing device and displays an image represented by the display image signal. The image display system includes a processor that acquires a medical image obtained by photographing a subject, detects a characteristic region from the medical image, and detects a characteristic region having a size that is an integral multiple of 2 or more of pixels constituting the medical image. A display unit grid in which display unit grids having a size exceeding the size of a processing unit for detecting a feature region are arranged in a medical image is set in a medical image, and a display unit grid with a display unit grid having a size exceeding the processing unit size for detecting a feature region is set in a medical image, and a feature region is The display is an image display system that generates a display signal that displays a display frame having one or more display unit grids corresponding to the size of the medical image in a superimposed manner on a characteristic region, and displays the display frame in a superimposed manner on a medical image.

An image processing method according to the present disclosure acquires a medical image obtained by photographing a subject, detects a characteristic region from the medical image, and detects a characteristic region having a size that is an integral multiple of 2 or more of pixels constituting the medical image. A display unit grid in which display unit grids, which are display unit grids and whose size exceeds the processing unit size for feature region detection, is arranged is set in a medical image, and a display unit grid is set in a medical image, and a display unit grid is arranged in a display unit grid whose size exceeds the processing unit size for feature region detection. This is an image processing method that generates a display signal that causes a display frame having one or more display unit grids corresponding to the size to be displayed in a superimposed manner on a feature region.

A program according to the present disclosure provides a computer with a function for acquiring a medical image obtained by photographing a subject, a function for detecting a characteristic region from a medical image, and a function for detecting a feature region from a medical image, and a size that is an integer multiple of 2 or more of the pixels constituting the medical image. A function to set a display unit grid in a medical image in which display unit grids having a size exceeding the size of a processing unit in detecting a feature region are arranged, and a display unit grid having a size exceeding the processing unit size in detecting a feature region, and a display unit grid arranged at a position corresponding to the position of the feature region. , is a program that realizes a function of generating a display signal that causes a display frame having one or more display unit grids corresponding to the size of a feature region to be displayed in a superimposed manner on the feature region.

According to the present invention, a feature region detected from a medical image has a size that is an integral multiple of 2 or more of the pixels constituting the medical image, and has a size that exceeds the size of a processing unit of the medical image. By overlapping display frames having one or more display unit grids, detection results can be displayed without impairing the reliability of detecting feature regions.

FIG. 1 is a functional block diagram of a medical image display system according to an embodiment. FIG. 2 is a flowchart showing the procedure of the medical image processing method according to the embodiment. FIG. 3 is a schematic diagram of a display grid set for a medical image. FIG. 4 is an explanatory diagram of a mode in which the medical image and the display grid are aligned using the center of gravity of the subject. FIG. 5 is an explanatory diagram of an example of the correspondence between the center of gravity of a three-dimensional image and the center of gravity of a tomographic image. FIG. 6 is an explanatory diagram of another example of the correspondence between the center of gravity of a three-dimensional image and the center of gravity of a tomographic image. FIG. 7 is an explanatory diagram showing an example of a display grid setting screen. FIG. 8 is a schematic diagram of a display frame superimposed on a medical image. FIG. 9 is an explanatory diagram of a modification of the display frame. FIG. 10 is an explanatory diagram of the effects of the embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same reference numerals are given to the same components, and overlapping explanations are omitted as appropriate.

[Example of configuration of medical image display system]
[Overall configuration of medical image display system]
FIG. 1 is a functional block diagram of a medical image display system according to an embodiment. The medical image display system 10 includes a medical image processing device 12, a medical image storage device 18, and a medical image viewer device 20.

The medical image processing device 12 is a terminal device used by users in hospitals, testing labs, and the like. The medical image processing device 12 may be a computer. The medical image processing device 12 includes a processor 14 and a memory 16.

Memory 16 includes a program memory in which programs containing instructions to be executed by processor 14 are stored. Memory 16 may include a data memory in which various data are stored.

The medical image processing device 12 executes the program read out from the memory 16 by the processor 14, and performs a medical image acquisition function, a feature region detection function, a display grid setting function, an alignment function, a display frame setting function, a display image signal generation function, and It realizes various functions including display image signal transmission function.

Here, the term image may include the meaning of an image signal representing an image and image data representing an image. Also, the term generation may be interpreted as synonymous with terms such as creation and production. Additionally, the term transmitting a signal may include the output of a signal from a source of signal output. Note that the processor 14 described in the embodiment is an example of one or more processors.

The medical image acquisition function is a function to acquire a medical image to be processed from the medical image storage device 18 or the like. Acquiring medical images may include transforming medical images and generating new medical images. For example, when a two-dimensional tomographic image is generated using raw data acquired from the CT imaging device 28 or the like, this can be included in the concept of acquiring a two-dimensional tomographic image. Note that the term "tomographic image" may include the concept of a cross-sectional image.

The characteristic region may include a region treated as a characteristic in a medical image, such as a pulmonary nodule, a fracture, a subarachnoid hemorrhage, and a cerebral infarction. A plurality of characteristic regions may be detected in one medical image. A known method can be applied to extract the feature region.

The feature region detection function may generate a mask image of the feature region. A known method can be applied to generate the mask image. The processing unit for characteristic region detection may be pixels constituting a medical image, or may be a region including a plurality of pixels. Note that the medical image is illustrated as a tomographic image 100 in FIG. 3 and the like. The characteristic region is illustrated in FIG. 3 using the reference numeral 102.

The display lattice is a lattice applied when defining a display frame to be superimposed on the feature region. The display grid has a structure in which a plurality of display unit grids are arranged two-dimensionally or three-dimensionally. That is, a two-dimensional display grid can be applied to a two-dimensional image. A three-dimensional display grid may be applied to a three-dimensional image.

The display grid setting function includes setting the number of display unit grids in each dimension and setting the size of the display unit grid. The size of the display unit grid is a positive integer multiple of the size of the pixels constituting the medical image, and is larger than the size of the processing unit for feature region detection.

For example, when the processing unit for feature region detection is one pixel, the display unit grid is made up of two or more pixels. In the case of a two-dimensional display grid, the area of the display unit grid is applied as the size of the display unit grid. In the case of a three-dimensional display grid, the volume of the display unit grid is applied to the size of the display unit grid.

The positioning function is a function that performs positioning between the subject and the display grid in the medical image. An example of alignment is a mode in which the position of the center of gravity of the subject is aligned with the position of the center of gravity of the display grid.

The display frame is a frame surrounding the characteristic region, and is displayed superimposed on the medical image. Surrounding the feature region here may mean that the feature region is included within the display frame, or the display frame may intersect with the outer edge of the feature region.

In the case of a two-dimensional display grid, a two-dimensional shape is applied to the display frame. Further, in the case of a three-dimensional display grid, a three-dimensional shape is applied to the display frame.

The display image signal generation function is a function that generates a display image signal representing a display image to be displayed using the display 22. The display image signal includes a display image signal representing a medical image and a display image signal representing a display frame. The display image signal may include a display image signal representing a feature region such as a mask image and a display image signal representing a display grid.

The display image signal transmission function is a function of transmitting a display image signal representing an image to be displayed using the display 22 to the medical image viewer device 20. The display 22 displays medical images and the like based on the display image signal. Note that the display image signal described in the embodiments is an example of a display signal.

The medical image storage device 18 stores medical images to which additional information defined by the DICOM standard is added. The medical image may be raw data acquired using modalities such as the CT imaging device 28 and the MRI imaging device 30 that photograph the subject, or may be volume data generated from the raw data. The medical image storage device 18 may be a mass storage device. Note that DICOM is an abbreviation for Digital Imaging and Communication in Medicine.

The medical image viewer device 20 is used by a user to view medical images. Medical image viewer device 20 includes a display 22 and an input device 24 . The display 22 displays an image represented by the display image signal acquired from the medical image processing device 12. The display 22 can display medical images processed by the medical image processing device 12 and medical images stored in the medical image storage device 18 based on commands from the medical image processing device 12 .

The input device 24 transmits an input signal according to a user's operation to the medical image processing device 12. The input device 24 may be an operating member such as a keyboard, a mouse, or a joystick. A touch panel type display 22 may be applied, and the display 22 and the input device 24 may be integrated.

The medical image display system 10 is communicably connected to a modality such as a CT imaging device 28 via a network 26 . The network 26 may be a LAN (Local Area Network). The network 26 may be an in-house LAN in a hospital or the like. The network 26 may include an external network such as a hospital.

Modalities may include PET devices, ultrasound diagnostic devices, CR devices, and the like. Note that PET is an abbreviation for Positron Emission Tomography. CR is an abbreviation for Computed Radiography.

[Procedures of medical image processing method]
FIG. 2 is a flowchart showing the procedure of the medical image processing method according to the embodiment. In the medical image acquisition step S10, the processor 14 acquires a medical image to be processed from the medical image storage device 18 or the like.

Medical images may be acquired by acquiring data in a format that can be processed by the processor 14, or by acquiring data in any format and converting it into data in a format that can be processed by the processor 14. good. After the medical image acquisition step S10, the process advances to a feature region detection step S12.

In the feature region detection step S12, the processor 14 automatically detects feature regions from the acquired medical image. After the feature region detection step S12, the process proceeds to a display grid setting step S14. In the feature region detection step S12, the processor 14 may detect a feature region based on the specified type of feature.

In the medical image acquisition step S10, a medical image in which feature regions have been automatically detected may be acquired. In this embodiment, instead of the feature region detection step S12, the processor 14 acquires information on the feature region attached to the medical image.

In the display grid setting step S14, the processor 14 sets a display grid for the medical image. In the display grid setting step S14, the user can set the number of display unit grids and the size of the display unit grid in each dimension using the input device 24 shown in FIG.

The order of the feature region detection step S12 and the display grid setting step S14 may be switched, or both may be performed in parallel. After the display grid setting step S14, the process proceeds to the alignment step S16.

In the alignment step S16, the processor 14 aligns the subject in the medical image and the display grid. To align the subject in the medical image with the display grid, a method may be applied in which the center of gravity of the medical image and the center of gravity of the display grid are aligned.

The position of the center of gravity in a two-dimensional medical image such as a tomographic image can be defined using a two-dimensional orthogonal coordinate system. The position of the center of gravity in a three-dimensional medical image can be defined using a three-dimensional orthogonal coordinate system. After the alignment step S16, the process proceeds to a display frame setting step S18.

In the display frame setting step S18, the processor 14 sets a display frame according to the size and position of the feature region detected in the feature region detection step S12. In the display frame setting step S18, the processor 14 specifies one or more display unit grids that include the feature region, and sets the outline of the aggregate of the specified one or more display unit grids as a display frame.

If multiple feature regions are detected, processor 14 may set a display frame for each feature region. Further, the processor 14 can set one display frame for one feature region. That is, the processor 14 can set one or more display frames for each of the plurality of feature regions. Note that in a tomographic image, two or more feature regions that are separately visually recognized may be the same feature region.

On the other hand, the processor 14 can set one display frame for a plurality of feature regions having the same generation source. For example, when the type of characteristic region is bleeding, one display frame can be set for characteristic regions corresponding to multiple bleedings that have the same bleeding source. After the display frame setting step S18, the process proceeds to the display image signal generation step S20.

In the display image signal generation step S20, the processor 14 generates a display image signal representing a display image to be displayed on the display 22 shown in FIG. That is, the processor 14 generates a display image signal of the medical image and a display image signal of the display frame. The processor 14 may generate a display image signal of the display grid and a display image signal of the display unit grid. After the display image signal generation step S20, the process proceeds to the display image signal transmission step S22.

In the display image signal transmission step S22, the processor 14 transmits the display image signal generated in the display image signal generation step S20 to the medical image viewer device 20. After the display image signal transmission step S22, the processor 14 ends the procedure of the image processing method.

Based on the received display image signal, the medical image viewer device 20 uses the display 22 to display a display image in which a display frame surrounding the characteristic region is superimposed on the medical image. The medical image viewer device 20 may use the display 22 to display the mask image and display grid of the characteristic region in a superimposed manner on the medical image.

[Specific example of display grid and display frame]
[Display grid settings]
FIG. 3 is a schematic diagram of a display grid set for a medical image. The figure shows, as a medical image, a tomographic image 100 of a brain taken using a CT imaging device 28, in which subarachnoid hemorrhage has been detected as a characteristic region 102. On the tomographic image 100, a mask image 103, which is the detection result of the characteristic region 102, is displayed in a superimposed manner.

The feature region 102 may have a polygonal shape. That is, the planar shape of the outline of the feature region 102, which is obtained by connecting the representative points of pixels forming the edge of the feature region 102 using line segments, is a polygonal shape. The center of gravity of the pixel can be used as the representative point of the pixel. The contour of the feature region 102 may be subjected to image processing such as smoothing, and the contour of the feature region 102 may be configured using at least one of a curve and a line segment, so that the feature region 102 may have an arbitrary shape.

For the tomographic image 100 shown in FIG. 3, the processor 14 shown in FIG. The display grid 110 is set as follows. FIG. 3 illustrates a two-dimensional display grid 110 in which two-dimensional display unit grids 112 are arranged two-dimensionally for a tomographic image 100 that is a two-dimensional medical image. An example of the two-dimensional display unit grid 112 is a cell.

The number of divisions of the display grid 110 may be finer than 5x5, such as 7x7 and 9x9, or coarser than 5x5, such as 3x3. The number of divisions of the display grid 110 may be 2×2 or more. The number of divisions of the display grid 110 may not have an aspect ratio of 1, such as 3×4 or 9×16.

[Display grid alignment]
The display grid 110 may be aligned with the tomographic image 100 at any of its four corners. In the display grid 110 shown in FIG. 3, the upper left end position 111 is used as a reference for alignment. The processor 14 shown in FIG. 1 can align the tomographic image 100 and the display grid 110 by overlapping the upper left end position 101 of the tomographic image 100 and the upper left end position 111 of the display grid 110. The display grid 110 only needs to have a shape that surrounds the entire subject, and the display grid 110 does not need to surround the entire tomographic image 100.

FIG. 4 is an explanatory diagram of a mode in which the medical image and the display grid are aligned using the center of gravity of the subject area. The processor 14 shown in FIG. 1 extracts the object region 104 from the tomographic image 100, calculates the center of gravity 106 of the object region 104, superimposes the center of gravity 114 of the display grid 110 on the center of gravity 106 of the object region 104, and displays the image. Registration of grid 110 and tomographic image 100 may be performed.

FIG. 5 is an explanatory diagram showing the correspondence between the center of gravity of a three-dimensional image and the center of gravity of a tomographic image. Note that in FIG. 5, the thickness of the tomographic image 100A, etc. is illustrated in a simplified manner. For the three-dimensional image 120 shown in FIG. 5, voxels constituting the three-dimensional image 120 can be set as a three-dimensional display unit grid 123. The three-dimensional display grid 121 is configured by display unit grids 123 arranged three-dimensionally. The same applies to the three-dimensional image 120 shown in FIG.

The three-dimensional image 120 corresponding to the tomographic image 100 may be generated using raw data, or may be generated using the tomographic images 100A to 100E. The same applies to tomographic images 100A to 100E shown in FIG.

FIG. 5 shows a tomographic image 100A, a tomographic image 100B, a tomographic image 100C, a tomographic image 100D, and a tomographic image 100E that correspond to the three-dimensional image 120. Note that in the following description, tomographic images 100A to 100E may be collectively referred to as tomographic images 100.

In each of the tomographic images 100A to 100E, the center of gravity 124 of the subject 122 of the three-dimensional image 120 is projected, and the center of gravity 106 is defined. When the respective positions of the tomographic images 100A to 100E are aligned and overlapped, the positions of the respective centers of gravity 106 match. That is, positioning of the display grid 110 is performed collectively for each of the tomographic images 100A to 100E.

Reference numeral 126 indicates a line passing through the center of gravity 124 in the three-dimensional image 120. Reference numeral 108 is a line passing through the center of gravity 106 of each of the tomographic images 100A to 100E. The line 108 is orthogonal to each of the tomographic images 100A to 100E.

FIG. 6 is an explanatory diagram of another example of the correspondence between the center of gravity of a three-dimensional image and the center of gravity of a tomographic image. In the tomographic image 100A shown in FIG. 6, a center of gravity 106A is defined based on the subject area 104A. Similarly, each of the tomographic images 100B to 100E is based on the subject area 104B, subject area 104C, subject area 104D, and subject area 104E, and the center of gravity 106B, center of gravity 106C, center of gravity 106D, and center of gravity 106E. is defined.

For each of the tomographic images 100A to 100E shown in FIG. 6, alignment of the display grid 110 is performed individually.

[Display grid settings]
FIG. 7 is an explanatory diagram showing an example of a display grid setting screen. The setting screen 200 shown in the figure is displayed using the display 22 shown in FIG. The user can view the settings screen 200 displayed using the display 22 and operate the input device 24 to set the parameters of the display grid.

The setting screen 200 includes a medical image display area 202 and a parameter display area 204. Medical images are displayed in the medical image display area 202. FIG. 7 shows a tomographic image 100 of the brain shown in FIG. 3 etc. as a medical image. Note that in FIG. 7, illustration of the center of gravity 106 of the tomographic image 100 and the center of gravity 124 of the display grid 110 shown in FIG. 4 etc. is omitted.

The parameter display area 204 includes a reference display area 206, a division number display area 208, and a feature area display area 210. The parameter display area 204 may include a button for switching between displaying and non-displaying information to be displayed in the medical image display area 202.

In the reference display area 206, a reference for positioning the tomographic image 100 and the display grid 110 is displayed. The reference display area 206 may display information input by the user using the input device 24 shown in FIG. Processor 14 may apply user-input criteria to perform alignment of tomographic image 100 and display grid 110.

When the processor 14 aligns the tomographic image 100 and the display grid 110 using a predefined alignment reference, the predefined alignment reference is displayed in the reference display area 206. The user may use the input device 24 to change the predefined alignment criteria.

In the division number display area 208, the number of divisions of the display grid 110 is displayed. FIG. 7 illustrates the number of divisions in the two-dimensional display grid 110. The division number display area 208 can display information input by the user using the input device 24 shown in FIG. Processor 14 may configure display grid 110 using the number of divisions entered by the user.

When the processor 14 sets the display grid 110 using a predefined number of divisions, the number of divisions display area 208 displays the value of the number of divisions defined in advance. The user can use the input device 24 to change the predefined number of divisions.

In the characteristic region display area 210, the type of the characteristic region 102 is displayed. The processor 14 displays the type of feature region 102 to be detected in the feature region display area 210. Processor 14 can set the number of divisions of display grid 110 depending on the type of feature region 102. In other words, the processor 14 can set the size of the display unit grid 112 depending on the type of feature region 102.

The processor 14 can set a reference for alignment between the tomographic image 100 and the display grid 110, depending on the type of the feature region 102, the number of divisions of the display grid 110, and the like. The reference display area 206, the number of divisions display area 208, and the feature area display area 210 may be configured in such a manner that a pull-down menu is applied to select any option from a plurality of predefined options.

The alignment between the three-dimensional image 120 and the display grid 121 shown in FIGS. 5 and 6 can be performed in the same way as the alignment between the two-dimensional tomographic image 100 and the two-dimensional display grid 110.

[Generation of display frame]
FIG. 8 is a schematic diagram of a display frame superimposed on a medical image. In the tomographic image 100 shown in the figure, a display frame 116 is superimposed. The display frame 116 represents the outline of a set of a plurality of display unit grids 112 including the feature region 102.

The display frame 116 may apply the contours of all display unit grids 112 including the feature region 102. The lines forming the display frame 116 can have any thickness and any color. The lines forming the display frame 116 may be treated in a different manner from the lines forming the display grid 110 and the display unit grid 112.

Hatching may be applied to the inside of the display frame 116. The hatching applied inside the display frame 116 may be a pattern such as dot hatching, or may be filled with any color. When a fill is applied as hatching inside the display frame 116, it is preferable to use a semi-transparent fill that allows the tomographic image 100 to pass through.

When the display frame 116 is superimposed on the tomographic image 100, the lines representing the outline of the display grid 110 and the lines representing the outline of the display unit grid 112 may be hidden. Furthermore, when the display frame 116 is superimposed on the tomographic image 100, a line representing the outline of the display unit grid 112 inside the display frame 116 is displayed, and a line representing the outline of the display unit grid 112 inside the display frame 116 is displayed. may be hidden. Furthermore, when the display frame 116 is superimposed on the tomographic image 100, the mask image 103 may be hidden.

The processor 14 may have a function of switching between display and non-display of the mask image 103, the outline of the display grid 110, the display unit grid 112, and the like. The processor 14 may switch between displaying and non-displaying the mask image 103 and the like based on a user's input using the input device 24. The processor 14 may use the display 22 to display a screen for switching between displaying and non-displaying the mask image 103 and the like.

The three-dimensional display frame in the three-dimensional image 120 shown in FIGS. 5 and 6 can be set similarly to the two-dimensional display frame 116. Note that illustration of a three-dimensional display frame superimposed on the three-dimensional image 120 is omitted.

[Modified example of display frame]
FIG. 9 is an explanatory diagram of a modification of the display frame. The outline of the display frame 116A shown in FIG. 9 is located outside the outline of the set of display unit grids 112 that constitute the display frame 116A. Thereby, deterioration in visibility of the characteristic region 102 can be suppressed. Note that illustration of the characteristic region 102 is omitted in FIG. 9 .

[Operations and effects of the image processing device, image display system, and image processing method according to the embodiment]
The image processing device, image display system, and image processing method according to the embodiment can obtain the following effects.

[1]
FIG. 10 is an explanatory diagram of the effects of the embodiment. A tomographic image 300A to which a bounding box 304 that collectively encloses a plurality of feature regions 302 is applied to a tomographic image 300 with a brain as an object shown in FIG. It is difficult to know whether 302 exists.

Furthermore, in the tomographic image 300B in which the mask image 306 corresponding to the feature region 302 is superimposed, it is difficult to comprehensively emphasize a plurality of feature regions 302, and there is a concern that a feature region 302 as small as several pixels may be overlooked. .

Furthermore, in the tomographic image 300C in which a plurality of bounding boxes 304 are applied to each isolated feature region 302, it is difficult to grasp the feature region 302 as a whole, and it is also difficult to grasp each feature region 302. Furthermore, if the source of bleeding is one, there is a desire to display it using one bounding box 304.

In contrast, the medical image display system 10 according to the present embodiment sets a display grid 110 on a medical image such as a tomographic image 100, as shown in FIG. In the display grid 110, display unit grids 112 larger than the processing unit size of the feature region 102 are arranged. The tomographic image 100 is a display frame 116 surrounding the feature region 102, and the display frame 116 using one or more display unit grids 112 is displayed in a superimposed manner. Thereby, the detection results of the feature region 102 can be displayed without impairing the reliability of detection of the feature region 102.

Furthermore, in an embodiment in which one display frame 116 is applied to a plurality of characteristic regions 102, if the source of bleeding is in one place, the request to display it using one bounding box is satisfied. obtain.

[2]
The display frame 116 has a different aspect from the lines that make up the display grid 110 and the lines that make up the display unit grid 112. For example, the thickness, color, and type of lines that make up the display unit grid 112 are made different from the lines that make up the display grid 110. This allows the display frame 116 to stand out more than the display grid 110 and the display unit grid 112.

[3]
In the tomographic image 100 on which the display frame 116 is superimposed, the display unit grid 112 is hidden. This makes it possible to make the display frame 116 stand out.

[4]
A mask image 103 is superimposed on the tomographic image 100 . Thereby, the display frame 116 and the mask image 103 are used together, and the visibility of the characteristic region 102 can be improved.

[5]
The size of the display unit grid is set according to the type of feature region. Thereby, the display frame 116 can be displayed according to the type of the feature region 102.

[6]
To align the tomographic image 100 and the display grid 110, the center of gravity 106 of the tomographic image 100 and the center of gravity 114 of the display grid 110 are applied. Thereby, accurate alignment between the tomographic image 100 and the display grid 110 can be performed.

[7]
A three-dimensional display grid 121 and a three-dimensional display frame can be set for the three-dimensional image 120. Thereby, the detection result of the feature region can be displayed without impairing the reliability of detecting the feature region in the three-dimensional image 120.

[8]
The feature region 102 can have an arbitrary shape. Thereby, the display frame 116 can be superimposed on the feature region 102 having an arbitrary shape.

[Hardware configuration of each processing unit and control unit]
The hardware structure of the processing unit that executes the processing of the medical image display system 10 and the medical image processing device 12 described in the above embodiments is various types of processors. Various types of processors include CPUs (Central Processing Units), PLDs (Programmable Logic Devices), ASICs (Application Specific Integrated Circuits), and the like.

A CPU is a general-purpose processor that executes programs and functions as various processing units. A PLD is a processor whose circuit configuration can be changed after manufacturing. An example of a PLD is an FPGA (Field Programmable Gate Array). An ASIC is a specialized electrical circuit that has circuitry specifically designed to perform a specific process.

One processing unit may be composed of one of these various types of processors, or may be composed of two or more processors of the same type or different types. For example, one processing unit may be configured using a plurality of FPGAs or the like. One processing unit may be configured by combining one or more FPGAs and one or more CPUs.

Furthermore, a plurality of processing units may be configured using one processor. As an example of configuring a plurality of processing units using one processor, there is a configuration in which one or more CPUs and software are combined to configure one processor, and one processor functions as a plurality of processing units. Such forms are typified by computers such as client terminal devices and server devices.

As another configuration example. One example is the use of a processor that implements the functions of the entire system including a plurality of processing units using one IC chip. Such a form is typified by a system on chip. Note that IC is an abbreviation for Integrated Circuit. Further, the system on chip is sometimes written as SoC using the abbreviation "System On Chip".

In this way, various processing units are configured using one or more of the various processors described above as a hardware structure. Furthermore, the hardware structure of various processors is, more specifically, an electric circuit (circuitry) that is a combination of circuit elements such as semiconductor elements.

[Example of application to program]
A program can be configured that causes a computer to implement various functions of the medical image display system 10 and the medical image processing apparatus 12 and each step of the image processing method described in this specification. For example, a computer is made to implement processes corresponding to the medical image acquisition function, feature region detection function, display grid setting function, alignment function, display frame setting function, display image signal generation function, and display image signal transmission function shown in FIG. A program can be configured.

In the embodiments of the present invention described above, constituent elements can be changed, added, or deleted as appropriate without departing from the spirit of the present invention. The present invention is not limited to the embodiments described above, and many modifications can be made within the technical idea of the present invention by those having ordinary knowledge in the field. Furthermore, the embodiments, modifications, and application examples may be implemented in appropriate combinations.

10 Medical image display system 12 Medical image processing device 14 Processor 16 Memory 18 Medical image storage device 20 Medical image viewer device 22 Display 24 Input device 26 Network 28 CT imaging device 30 MRI imaging device 100 Tomographic image 100A Tomographic image 100B Tomographic image 100C Tomographic image Image 100D Tomographic image 100E Tomographic image 102 Feature area 103 Mask image 104 Subject area 104A Subject area 104B Subject area 104C Subject area 104D Subject area 104E Subject area 106 Center of gravity 106A Center of gravity 106B Center of gravity 106C Center of gravity 106D Center of gravity 106E Center of gravity 10 8 Line 110 Display grid 112 Display unit grid 114 Center of gravity 116 Display frame 116A Display frame 120 Three-dimensional image 121 Display grid 122 Subject 123 Display unit grid 124 Center of gravity 126 Line 200 Setting screen 202 Medical image display area 204 Parameter display area 206 Reference display area 208 Division number display area 210 Feature area display area 300 Tomographic image 300A Tomographic image 300B Tomographic image 300C Tomographic image 302 Feature area 304 Bounding box 306 Mask images S10 to S24 Each step of the image processing method

Claims (15)

An image processing device comprising one or more processors,
The processor includes:
Obtain medical images obtained by photographing the subject,
detecting a feature region from the medical image;
A display unit grid having a size that is an integer multiple of 2 or more of the pixels constituting the medical image, and a display unit grid in which display unit grids having a size exceeding a processing unit size in detecting the feature region are arranged. set to the medical image;
An image processing device that generates a display signal that causes a display frame having one or more of the display unit grids corresponding to the size of the feature region to be displayed superimposed on the feature region at a position corresponding to the feature region. The processor includes:
The image processing apparatus according to claim 1, wherein the display signal for displaying the outline of the display frame is generated using a mode different from the outline of the display unit grid. The processor includes:
The image processing apparatus according to claim 2, wherein the display signal is generated to display the outline of the display frame outside the outline of the display unit grid serving as an edge of the feature area. The processor includes:
The image processing apparatus according to claim 2 or 3, wherein the display signal that makes the display unit grid non-display is generated. The processor includes:
generating a mask image corresponding to the feature region;
The image processing device according to any one of claims 1 to 4, which generates the display signal representing the mask image. The processor includes:
The image processing device according to any one of claims 1 to 5, wherein the size of the display unit grid is set depending on the type of the feature area. The processor includes:
The image processing device according to any one of claims 1 to 6, wherein the medical image and the display grid are aligned using a center of gravity of the medical image and a center of gravity of the display grid. The processor includes:
The image processing device according to any one of claims 1 to 7, wherein the medical image and the display grid are aligned using a center of gravity of a subject region in the medical image and a center of gravity of the display grid. The processor includes:
The image processing apparatus according to any one of claims 1 to 8, wherein the display grid is set in which the two-dimensional display unit grids are arranged in a two-dimensional manner. The processor includes:
The image processing apparatus according to any one of claims 1 to 8, wherein the display grid is configured by arranging the three-dimensional display unit grids in a three-dimensional manner. The processor includes:
The image processing device according to any one of claims 1 to 10, wherein the feature area is detected as a polygon. an image processing device comprising one or more processors;
a display that receives a display image signal transmitted from the image processing device and displays an image represented by the display image signal;
An image display system comprising:
The processor includes:
Obtain medical images obtained by photographing the subject,
detecting a feature region from the medical image;
A display unit grid having a size that is an integer multiple of 2 or more of the pixels constituting the medical image, and a display unit grid in which display unit grids having a size exceeding a processing unit size in detecting the feature region are arranged. set to the medical image;
generating a display signal that causes a display frame having one or more of the display unit grids corresponding to the size of the feature region to be displayed superimposed on the feature region at a position corresponding to the feature region;
The display is an image display system that displays the display frame superimposed on the medical image. Obtain medical images obtained by photographing the subject,
detecting a feature region from the medical image;
A display unit grid having a size that is an integer multiple of 2 or more of the pixels constituting the medical image, and a display unit grid in which display unit grids having a size exceeding a processing unit size in detecting the feature region are arranged. set to the medical image;
An image processing method that generates a display signal that causes a display frame having one or more of the display unit grids corresponding to the size of the feature region to be displayed at a position corresponding to the position of the feature region in a superimposed manner on the feature region. to the computer,
A function to obtain medical images obtained by photographing a subject;
a function of detecting a feature region from the medical image;
A display unit grid having a size that is an integer multiple of 2 or more of the pixels constituting the medical image, and a display unit grid in which display unit grids having a size exceeding a processing unit size in detecting the feature region are arranged. a function to be set in the medical image; and a display signal that causes a display frame having one or more display unit grids corresponding to the size of the characteristic region to be displayed in a position corresponding to the position of the characteristic region, superimposed on the characteristic region. A program that realizes the function of generating . A non-transitory computer-readable recording medium, on which the program according to claim 14 is recorded.

Images