CN115346032A - Display method, device, equipment and storage medium - Google Patents

Abstract

According to the display method, the display device, the display equipment and the storage medium, the first cutting plane is determined and displayed based on the operation of a user, the second cutting plane is determined according to the first cutting plane, the three-dimensional model cutting plane corresponding to the second cutting plane is displayed, the three-dimensional model cutting plane is the cutting plane corresponding to the three-dimensional model after being cut by the second cutting plane, the second cutting plane is coincident with or parallel to the first cutting plane, the two-dimensional image cutting plane corresponding to the second cutting plane is generated according to medical image data, and the two-dimensional image cutting plane is displayed at the position of the three-dimensional model cutting plane in a correlated mode, so that the user can conveniently and correspondingly view any cutting plane and the corresponding two-dimensional image cutting plane of the three-dimensional model, and the use experience of the user is improved.

Description

Display method, device, equipment and storage medium

technical field

The present invention relates to the technical field of three-dimensional image display, in particular to a display method, device, equipment and storage medium.

Background technique

In current 3D model display applications, it is usually necessary to display a section of the 3D model, so that users can observe the 3D model more clearly and intuitively.

Among them, when displaying the section of the 3D model, only looking at the section of the 3D model usually cannot meet the needs of users. For example, when a doctor checks the section of the 3D model of a tissue or organ under a certain section plane, he usually cannot determine the condition well. You also need to check the 2D image slice corresponding to the slice.

How to display the 3D model slices and 2D image slices so that users can view them is an urgent problem to be solved.

Contents of the invention

The present invention provides a display method, device, equipment and storage medium to solve the problem in the prior art that it is inconvenient for users to view the display results when displaying the three-dimensional model slices and the two-dimensional image slices.

In a first aspect, the present invention provides a display method, the method comprising:

determining and displaying the first section plane based on the user's operation;

Determine the second sectional plane according to the first sectional plane, and display the 3D model sectional plane corresponding to the second sectional plane; the 3D model sectional plane is the sectional plane corresponding to the 3D model after being cut by the second sectional plane ; The second sectional plane coincides with or is parallel to the first sectional plane;

A two-dimensional image section corresponding to the second section is generated according to the medical image data, and the two-dimensional image section is associated and displayed at the three-dimensional model section.

Optionally, associating and displaying the 2D image slices at the 3D model slices includes:

superimposing and displaying the 2D image section on the 3D model section;

The positions of the same tissue or organ in the two-dimensional image slice and the three-dimensional model slice are respectively determined, and the two positions are set to the same color.

Optionally, generating a two-dimensional image section corresponding to the second section according to the medical image data includes:

determining a volume box according to the maximum value in each direction in the medical image data and displaying the volume box;

determining the intersection line position between the second section plane and the volume box, and determining the boundary position of the two-dimensional image section plane in the medical image data according to the intersection line position, and determining the boundary position according to the boundary position 2D image slices.

Optionally, associating and displaying the 2D image slices at the 3D model slices includes:

setting the distance between the 2D image slice and the 3D model slice as a preset distance;

The corresponding tissues or organs in the 2D image section and the 3D model section are connected by a projection line.

Optionally, the method also includes:

Receive a user's transparent processing operation instruction on the two-dimensional image section, and perform transparent processing on the two-dimensional image section according to the instruction; the transparent processing operation instruction includes a transparency percentage.

Optionally, the first section plane is determined and displayed based on user operations, including:

After the user rotates the three-dimensional model to the orientation of interest, determine a triangular patch selected by the user on the surface of the three-dimensional model; determine the first cutting plane according to the plane where the triangular patch is located, and displaying said first cut plane;

Alternatively, display the section view angle, when the user rotates the 3D model to the orientation of interest or the user rotates the section view angle to the interest angle of view, any plane perpendicular to the section view angle is determined as first cut plane, and displays the first cut plane.

Optionally, the first section plane is determined and displayed based on user operations, including:

Determine at least one surgical cutting plan corresponding to the three-dimensional model, and determine at least one first section plane according to the surgical cutting plan; the surgical cutting plan includes cutting planes and/or surgical positions; or, determine the three-dimensional model type, at least one first section plane is determined on the anatomical plane or section section corresponding to the type of the three-dimensional model;

Display the first cut plane.

Optionally, after determining the first section plane, the method further includes:

storing the determined first section plane, and displaying a list containing the first section plane on the screen;

An operation instruction for the user to select a first section plane from the list is received, and a selected first section plane is determined and displayed according to the operation instruction.

Optionally, determining at least one surgical cutting scheme corresponding to the three-dimensional model includes:

Identifying lesions according to the three-dimensional model or medical image data used to construct the three-dimensional model;

The surgical cutting scheme is determined based on at least one of information on the shape, size and location of the lesion.

Optionally, the method also includes:

According to the determined first section plane, the three-dimensional model is adjusted to an orientation in which the first section plane is parallel to the screen.

Optionally, determining the second section plane according to the first section plane includes:

The second section plane is determined according to a user-triggered drag operation on the first section plane.

In a second aspect, the present invention provides a display device, the device comprising:

A first determining module, configured to determine and display a first section plane based on user operations;

The second determining module is configured to determine a second section plane according to the first section plane, and display a 3D model section corresponding to the second section section; the 3D model section section is the 3D model passing through the second The corresponding cut plane after cutting the cut plane; the second cut plane coincides with or is parallel to the first cut plane;

The display module is configured to generate a two-dimensional image slice corresponding to the second slice according to the medical image data, and display the two-dimensional image slice in association with the three-dimensional model slice.

In a third aspect, the present invention provides an electronic device, comprising: at least one processor and a memory;

the memory stores computer-executable instructions;

At least one processor executes computer-implemented instructions stored in the memory, such that the at least one processor performs the method according to any one of the first aspect.

In a fourth aspect, the present invention provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the method according to any one of the first aspect is implemented.

A display method, device, device, and storage medium provided by the present invention determine and display a first cut plane based on user operations, determine a second cut plane according to the first cut plane, and display the same as the second cut plane. The cut plane of the three-dimensional model corresponding to the cut plane, the cut plane of the three-dimensional model is the cut plane corresponding to the three-dimensional model after being cut by the second cut plane, the second cut plane coincides with or is parallel to the first cut plane, according to medical The image data generates a two-dimensional image section corresponding to the second section plane, and displays the two-dimensional image section in association with the three-dimensional model section, so that the user can conveniently view any section of the three-dimensional model and the corresponding Two-dimensional image section to improve user experience.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.

FIG. 1 is a diagram of an application scenario provided by an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a display method provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of an associated display of a three-dimensional model section and a two-dimensional image section provided by an embodiment of the present invention;

Fig. 4 is a schematic diagram of verifying the slice position of a two-dimensional image based on a volume box provided by an embodiment of the present invention;

Fig. 5 is a schematic diagram of determining a first section plane provided by an embodiment of the present invention;

Fig. 6 is another schematic diagram of determining the first section plane provided by the embodiment of the present invention;

Fig. 7 is another schematic diagram of determining the first section plane provided by the implementation of the present invention;

Fig. 8 is a schematic diagram showing a liver model provided by an embodiment of the present invention;

Fig. 9 is a schematic diagram showing a human body model provided by an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a display device provided by an embodiment of the present invention;

FIG. 11 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present invention.

By way of the above drawings, specific embodiments of the invention have been shown and will be described in more detail hereinafter. These drawings and written descriptions are not intended to limit the scope of the inventive concept in any way, but to illustrate the inventive concept for those skilled in the art by referring to specific embodiments.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention.

Fig. 1 is a diagram of an application scene provided by an embodiment of the present invention. As shown in Fig. 1, the execution subject of the display method of a three-dimensional model may be a display device, and the display device may be a product such as a computer, a handheld computer, or a mobile phone, for example, as shown in Fig. A computer 13 in 1. In the medical field, a three-dimensional model can be constructed based on medical imaging data (CT data) generated by a medical imaging device 10. The medical imaging device 10 includes a detector 11, a scanning bed 12 and a computer 13, and the scanning bed 12 moves to a position or In the process of moving at a constant speed, the detector 11 rotates 360 degrees around the scanning bed 12 to collect CT (Computed Tomography) data of the scanning target 20 (human body), and the computer 13 needs to control the movement of the scanning bed 12 and obtain CT data. The computer for reconstructing and displaying the three-dimensional model and the computer for controlling the movement of the scanning bed may be the same computer or different computers.

When the user browses the 3D model, it is necessary to observe the 3D model section and the 2D image section under a certain section plane at the same time. The 3D model section and the 2D image section are different images for the same section plane, and the 3D model section is A sectional view corresponding to a section plane, and a two-dimensional image section plane is an image map (CT image) corresponding to the section plane. In the prior art, when displaying a 3D model, the 3D model section and the 2D image section are displayed in different areas, and the user cannot quickly switch between the 3D model section and the 2D image section to view the 3D model section and the 2D image section. Dimensional image slices.

Based on the above problems, when displaying the 3D model section and the 2D image section, the present application displays the 2D image section by associating the 2D image section at the 3D model section, that is, the different parts of the 2D image section and the 3D model section respectively. Different parts are associated and displayed separately, so that users can easily view the status of any tissue or organ in the 3D model section in the 2D image section, and facilitate the user's conversion between 2D space and 3D space.

Fig. 2 is a schematic flowchart of a display method provided by an embodiment of the present invention. The method is applied to a display device. As shown in Fig. 2, the method includes steps S201 to S203:

Step S201 , determining and displaying a first cut plane based on a user's operation.

When displaying the 3D model slices and the corresponding 2D image slices, the created 3D model can be loaded first, or a 3D model can be created according to the medical image data, and the 3D model can be displayed.

Specifically, the three-dimensional model can be generated based on medical imaging data, and the medical imaging data can be generated based on the medical imaging device 10 shown in FIG. Correspondingly, the medical image data may be CT data, ultrasound data or MRI (Magnetic resonance imaging) data.

After the medical image data is acquired, the acquired medical image data can be used to reconstruct the three-dimensional model based on the modeling software. The principle of 3D model reconstruction is: different tissues correspond to different medical image values, and the data of different tissues are segmented from the medical image data by using tools such as threshold segmentation and mask editing in the modeling software. After extracting the data of different tissues, 3D models of different tissues can be established, and when there are multiple extracted tissues, the 3D models of multiple tissues are fused to generate the final 3D model, such as the generated 3D model includes Liver, arteries, veins and lesions, etc.

When displaying the 3D model cut plane and the corresponding 2D image cut plane, the cut plane needs to be determined first, specifically, the first cut plane can be determined first. The first cut plane may be determined based on a user's operation. The user's operation may refer to rotating the 3D model during the process of browsing the 3D model, or clicking a triangle patch on the 3D model, or selecting a first cutting plane in the list of the first cutting planes, the first The list of section planes may be generated after the first section plane is determined and saved during the browsing process, or may be generated after the first section plane is determined and saved before browsing the three-dimensional model.

When determining the first section plane in the process of browsing the 3D model, it can be determined and displayed, or multiple first section planes can be determined and saved during the browsing process, and can be selected when observation is required. A first cut plane is displayed.

When the first section plane is determined before viewing the three-dimensional model, a plurality of first section planes may be pre-determined and saved. When displaying the three-dimensional model, receiving an instruction from the user to select a first section plane from a plurality of first section planes, and displaying the first section plane.

Step S202: Determine a second section plane according to the first section plane, and display a 3D model section plane corresponding to the second section section plane; the 3D model section section is the 3D model after being cut by the second section section plane Corresponding cut plane: the second cut plane coincides with or is parallel to the first cut plane.

After the first section plane is determined, the second section plane can be further determined, and the second section plane coincides with or is parallel to the first section plane, that is, it is possible to view the three-dimensional images corresponding to different section planes under a certain section angle of view. Model section.

Optionally, determining the second section plane according to the first section plane includes: determining the second section plane according to a user-triggered drag operation on the first section plane.

When the second cut plane is parallel to the first cut plane, the second cut plane may be determined by dragging the first cut plane by the user.

In practice, the second section plane and the corresponding three-dimensional model section plane may be displayed in real time during the process of moving the first section plane. Wherein, the moving speed and moving distance of the first cut plane may be determined according to the user's dragging speed.

Optionally, the purpose of browsing the three-dimensional model input by the user may be obtained. When the purpose is to learn the structure of the three-dimensional model, the movement range of the determined section plane is the first range; when the purpose is to observe the In the case of the cutting plane, the determined moving range of the cutting plane is the second range; wherein, the first range is greater than the second range.

Specifically, when browsing the 3D model is used to learn and understand the specific structure of a certain tissue or organ, it is hoped that the 3D model cut plane corresponding to any cut plane in the 3D model can be viewed, and the movement range of the first cut plane can be relatively small. Large; when browsing the 3D model is used to observe the cutting plane in surgical planning, the user only pays attention to the cutting plane of the 3D model near the cutting plane, and can move within a small range near the cutting plane at this time, and the moving range is relatively large. Small. Alternatively, the user only pays attention to the cut plane of the three-dimensional model corresponding to the cut plane, and may not move the first cut plane, that is, the first cut plane coincides with the second cut plane.

After the second cutting plane is determined, the 3D model cut plane corresponding to the second cut plane can be displayed, the second cut plane only indicates the direction and position of the plane used to cut the 3D model, and the 3D model cut plane indicates that the 3D model passes through the The corresponding cross-sectional view after cutting by the second cutting plane.

Step S203, generating a 2D image slice corresponding to the second slice according to the medical image data, and displaying the 2D image slice in association with the 3D model slice.

A corresponding two-dimensional image section can also be generated for the second section plane, wherein the two-dimensional image section is also generated based on medical image data. Wherein, since the medical image data is determined by step-by-step scanning or helical scanning, no matter which scanning mode is used to obtain the medical image data, the two-dimensional image section of the scanning target cannot be viewed in any direction. Stepping scanning refers to the data of a cross-section collected when the scanning target moves to the imaging position and stops moving, and the detector rotates around the scanning table for a circle. This method can only view the two-dimensional image slices of the cross-section. Helical scanning refers to the data collected when the detector rotates around the scanning table when the scanning target moves at a constant speed. In this scanning mode, the starting point and end point of the detector's rotation are different, and the acquired data is not a cross-sectional data. , so it is necessary to interpolate the acquired data to obtain two-dimensional image slices on the cross section.

In order to generate 2D image slices corresponding to 3D model slices, it is necessary to perform multi-planar reconstruction of medical image data to generate 2D image slices at any angle or orientation. Specifically, when generating a two-dimensional image section at any angle or orientation, the slice direction can be determined first, wherein the slice direction is perpendicular to the second section plane, and then new medical image data is generated based on the slice direction and medical image data. The tomographic image of the image data is parallel to the second section plane, so as to obtain a two-dimensional image section corresponding to the second section plane.

As shown in Figure 3, when the 3D model cut planes corresponding to different positions (each second cut plane) on the sectional angle of view O are obtained by moving the first cut plane, the 3D model cut plane associated with the 3D model cut plane can be obtained according to the position of the second cut plane. 2D image section L, and display the 2D image section L in association with the 3D model section, that is, when the first section is moved to change the position of the first section, the 3D model section follows the first section The position of the 2D image changes according to the change of the position, so as to display the specific structure inside the 3D model under different first cut planes. At the same time, the cut plane L of the 2D image also changes with the change of the cut plane of the 3D model.

Linking and displaying the 2D image section at the 3D model section refers to associating the tissue or organ in the 3D model section with the corresponding tissue or organ in the 2D image section, because for non-professionals, the 2D image section is relatively simple. Abstraction, it is impossible to well associate the tissues or organs in the two-dimensional image section with the tissues or organs in the three-dimensional model section, and it is convenient for users to view by associating the same tissue or organ in the two.

By associating and displaying the 2D image slices at the 3D model slices, it is convenient for the user to simultaneously view the 3D model slices and the 2D image slices when viewing the 3D model, realizing simultaneous display of both in one window.

Optionally, when displaying the 3D model section and the 2D image section, one of them can also not be displayed according to the user's needs. For example, in order to facilitate the observation of the 3D model section, you can choose not to display the 2D image section; or, for the convenience of observation 2D image section, you can choose not to display the 3D model section. Alternatively, the 2D image section may not be displayed during the process of moving the first section section, but only the 3D model section is displayed, and when the 3D model section of interest is browsed, the 2D image section corresponding to the 3D model section is displayed.

A display method, device, device, and storage medium provided by the present invention determine and display a first section plane selected by the user; the three-dimensional model is generated based on medical image data, and the first section plane is triggered by the user The dragging operation of the cut plane determines the second cut plane, and displays the cut plane of the three-dimensional model corresponding to the second cut plane, and the cut plane of the three-dimensional model is the cut plane corresponding to the three-dimensional model after being cut by the second cut plane, according to The medical image data generates a two-dimensional image section corresponding to the second section plane, and associates and displays the two-dimensional image section at the section of the three-dimensional model, so that the user can conveniently view any section of the three-dimensional model and the corresponding two-dimensional image section to improve the user experience.

Optionally, associating and displaying the 2D image slices at the 3D model slices includes:

Overlaying and displaying the two-dimensional image slices on the three-dimensional model slices; respectively determining the positions of the same tissue or organ in the two-dimensional image slices and the three-dimensional model slices, and setting the two positions to the same color.

Among them, when performing associated display, the two-dimensional image section can be superimposed and displayed on the three-dimensional model section, so as to facilitate the direct correspondence of the same tissue or organ. In order to further distinguish different tissues or organs, for the same tissue or organ, the same tissue or organ may be shown in the same color on the three-dimensional model section and the two-dimensional image section.

Specifically, the position of the same tissue or organ in the two-dimensional image slice and the three-dimensional model slice can be determined first, and the two positions are set to the same color. For example, the liver in both the 2D image slice and the 3D model slice is set to red.

By setting the same color for the same tissue or organ in the two-dimensional image section and the three-dimensional model section, it is convenient for the user to accurately determine the corresponding part in the two section images according to the color, and the user's experience when viewing the three-dimensional model is improved.

Optionally, generating a two-dimensional image section corresponding to the second section according to the medical image data includes:

Determine the volume box according to the maximum value in each direction in the medical image data and display the volume box; and determine the boundary position of the two-dimensional image slice in the medical image data according to the position of the intersection line, according to the described The boundary position determines the slice of the 2D image.

In the process of determining the two-dimensional image section, the volume box can also be determined according to the maximum value of the medical image data in each direction, and the volume box and the three-dimensional model can be displayed at the same time. A schematic diagram of the volume box verifying the position of the cut plane of the two-dimensional image, as shown in Figure 4, the volume box V can be displayed in the form of a transparent wireframe. After the second section plane is determined, the position of the intersection line between the second section plane and the volume box can be determined, so that the boundary position of the two-dimensional image section plane in the medical image data can be determined according to the position of the intersection line, that is, the boundary position formed by the boundary position The plane is the same as the plane formed by the position of the intersection line. After the boundary position in the medical image data is determined, the medical image data corresponding to the plane passing through the boundary position is a two-dimensional image section, so that the two-dimensional image section can be accurately determined.

After the 2D image slice is determined, the 2D image slice can be positioned in the volume box by making each edge of the 2D image slice coincide with each intersection line, and the 2D image slice overlaps with the 3D model slice, which improves the Positioning speed and accuracy of 2D image slices.

Optionally, associating and displaying the 2D image slices at the 3D model slices includes:

The distance between the 2D image section and the 3D model section is set as a preset distance; the corresponding tissues or organs in the 2D image section and the 3D model section are connected by a projection line.

When the 2D image slices and the 3D model slices are displayed in association, they can also be displayed at certain intervals. The distance between the 3D model cut plane and the 2D image cut plane is determined as a preset distance, and the preset distance may be a preset value or a value input by the user. When it is a preset value, there is no limitation on the size of the value.

The associated display of the tissue or organ in the 3D model section and the 2D image section can be realized by setting a projection line, which is used to connect the tissue or organ in the 3D model section and the corresponding data part in the 2D image section. Optionally, the projection lines may be arranged along the contours of the tissues or organs, or the color of the projection lines may be the same as the color of each tissue or organ in the section plane of the three-dimensional model.

By setting the projection line, the same tissue or organ in the 2D image section and the 3D model section can be associated. At the same time, the 2D image section and the 3D model section can be displayed at a certain distance, allowing users to view separately and simultaneously view 3D Requirements for model slices and 2D image slices.

Optionally, the method also includes:

Receive a user's transparent processing operation instruction on the two-dimensional image section, and perform transparent processing on the two-dimensional image section according to the instruction; the transparent processing operation instruction includes a transparency percentage.

In order to facilitate the observation of 2D image slices and 3D model slices, the 2D image slices can also be transparentized. When the 2D image slices and 3D model slices are overlapped and displayed, further, the 2D image slices can be made transparent Processing, through the transparent processing of the 2D image section, it is convenient for users to see the 3D model section.

In addition, in order to facilitate the user's viewing, different transparency percentages can be set according to the user's viewing habits.

The transparent processing of the two-dimensional image slices can be realized by performing the transparent processing operation instruction, so that the user can view the two-dimensional image slices and the three-dimensional model slices at the same time.

The method for determining the first cutting plane in the process of browsing the three-dimensional model will be described in detail below. Among them, in the prior art, the browsing software of the 3D model usually only establishes the section plane based on the X-axis, Y-axis and Z-axis directions, and the user can drag the section plane to view the internal structure of the 3D model under the axial perspective. However, the axial viewing angle cannot meet the user's needs for different viewing angles of the 3D model. The following provides two methods for creating a section plane different from the axial perspective, which reduces the complexity of creating a section plane for users and improves efficiency at the same time.

Optionally, the first section plane is determined and displayed based on user operations, including:

After the user rotates the three-dimensional model to the orientation of interest, determine a triangular patch selected by the user on the surface of the three-dimensional model; determine the first cutting plane according to the plane where the triangular patch is located, and displaying said first cut plane;

Alternatively, display the section view angle, when the user rotates the 3D model to the orientation of interest or the user rotates the section view angle to the interest angle of view, any plane perpendicular to the section view angle is determined as first cut plane, and displays the first cut plane.

Fig. 5 is a schematic diagram of determining the first section plane provided by the embodiment of the present invention. As shown in Fig. 5, the surface of the three-dimensional model M is composed of several triangular faces, and the user can browse through the operation of rotating, moving and zooming 3D model, when the user is browsing the 3D model, if it is determined to adjust the 3D model M to the orientation of interest, the user can select a triangular patch on the surface of the 3D model, as indicated by the arrow in Figure 5 For the triangular surface, the first cutting plane P may be determined according to the plane where the triangular surface is located. That is, the first cutting plane is determined by the user clicking on the triangular patch of the three-dimensional model M.

Optionally, after the first section plane P is determined, it can be displayed on the display in the form of a wireframe to indicate the first section plane, such as the wireframe P shown in Figure 5; in addition, based on the selected A pointer perpendicular to the triangular face is generated and displayed on the display screen to indicate the cutting angle O corresponding to the first cutting plane.

Optionally, when determining the first cutting plane according to the plane where the triangular surface is located, the plane where the triangular surface is located can be determined as the first cutting plane, or it can be parallel to the triangular surface and have a certain The plane of the spacing is determined as the first cutting plane.

When the first cut plane is displayed, the wire frame of the first cut plane protrudes from the outline of the three-dimensional model M, so that the user can observe and confirm the selected first cut plane.

In addition, the first sectioning plane may also be determined by establishing a sectioning angle of view. FIG. 6 is another schematic diagram of determining the first section plane provided by the embodiment of the present invention. As shown in FIG. The cutting plane, that is, the cutting plane P is perpendicular to the cutting angle O.

The cutting angle O and the cutting plane P are fixedly set relative to the display screen. When the three-dimensional model M is rotated, translated or zoomed, the positional relationship between the cutting angle O and the cutting plane P relative to the three-dimensional model M changes, and the relative position of the three-dimensional model M can be obtained. For different cutting angles and cutting planes. Specifically, when the user rotates the three-dimensional model to the orientation of interest or rotates the viewing angle of cutting to the viewing angle of interest, the first cutting plane can be obtained. As shown in FIG. 6 , the cutting angle O may be a viewing angle perpendicular to the display screen, and the corresponding cutting plane P is parallel to the screen, so that the first cutting plane of the 3D model is always facing the user to facilitate browsing by the user.

Optionally, the positions of the established cutting angle O and the corresponding cutting plane P are variable, that is, the cutting angle O and the corresponding cutting plane P are not fixed, and the cutting angle O and the corresponding cutting plane P can be rotated or rotated relative to the display screen. Move, so that different first sectional planes can be obtained by rotating or moving the sectional angle O and the process of the sectional plane P. During the process of rotating or moving the sectional angle O and the sectional plane P, the positional relationship of the 3D model M relative to the sectional angle O and the sectional plane P changes, so as to obtain different first sectional planes.

Fig. 7 is another schematic diagram of determining the first section plane provided by the implementation of the present invention. As shown in Fig. 7, the section angle O and the section plane P perpendicular to each other can be displayed on the display screen, such as displaying the section in the form of a pointer. The cutting angle O displays the cutting plane P in the form of a wireframe. The cutting angle O and the cutting plane P can be rotated arbitrarily relative to the display screen. Specifically, it can be any one of the rotating cutting angle O or the cutting plane P. One will rotate with it. When the cutting angle O or the cutting plane P is rotated to the orientation of interest, the cutting plane at this time is the determined first cutting plane.

Through the above method, the first section plane can be obtained in any direction that the user is interested in in the process of browsing the 3D model, so as to obtain the corresponding 3D model section and 2D image section, improve the efficiency of creating the first section section, and at the same time reduce the the complexity.

In addition to determining the first section plane during browsing the 3D model, multiple section planes may be predefined before browsing the 3D model for the user to select.

Optionally, the first section plane is determined and displayed based on user operations, including:

Determine at least one surgical cutting plan corresponding to the three-dimensional model, and determine at least one first section plane according to the surgical cutting plan; the surgical cutting plan includes cutting planes and/or surgical positions; or, determine the three-dimensional model type, at least one first section plane is determined from the anatomical plane or section section corresponding to the type of the three-dimensional model; and the first section plane is displayed.

When the three-dimensional model M is a tissue or organ of the body, in order to improve the accuracy of the operation, doctors usually perform operation planning and operation simulation based on the three-dimensional model. Specifically, a surgical cutting plan may be generated according to the three-dimensional model and/or medical image data, or a surgical cutting plan formulated by a doctor may be received. Surgical cutting plan can include: cutting plane or surgical position.

For the cutting plane, doctors usually want to check whether the cutting range corresponding to the cutting plane is reasonable, therefore, the determined cutting plane can be determined as the first cutting plane, and correspondingly, the direction perpendicular to the first cutting plane is the cutting angle of view O . The number of cutting surfaces can be one or more.

In addition, the placement direction of the 3D model can also be adjusted according to the surgical position in the surgical cutting plan, and the side of the 3D model relative to the display screen can be adjusted to the surgical position, wherein any plane parallel to the display screen is the first cutting plane , the direction perpendicular to the first cutting plane is the cutting angle of view O, that is, the surgical angle of view. By viewing the cut plane of the 3D model under the surgical angle of view, some risks that may be encountered during the operation can be ruled out in advance, so as to optimize the operation before the operation. cutting scheme.

Alternatively, when determining the first section plane, first determine the type of the three-dimensional model, and determine the anatomical planes or segmental sections corresponding to the type of the three-dimensional model as multiple first section planes.

To facilitate the understanding of the internal structure of a tissue or organ, some organs or tissues have medical anatomical planes or medical segments, for example, long-axis and short-axis planes of the heart, standard segments of the lungs, and standard segments of the liver Etc., for example, the long-axis plane of the heart is determined based on the direction of the apex, mitral valve, and interventricular septum, and the segmentation of the liver is determined based on the distribution of blood vessels in the liver. In order to browse these anatomical planes or segmented sections conveniently in the process of browsing the 3D model, it is also possible to determine the type of tissue or organ in the 3D model, and then determine the first section plane according to the determined type, that is, the determined type The corresponding anatomical plane or segmental section is determined as the first cutting plane. Wherein, the anatomical plane or segmented section is determined based on certain fixed features in the tissue or organ, therefore, the establishment of the anatomical plane or segmented section can be established manually by the user or by an algorithm. After the anatomical plane and segmented section are determined, they can be stored as the first section plane, so that multiple first section planes can be determined.

Fig. 8 is a schematic diagram showing a liver model provided by an embodiment of the present invention. As shown in Fig. 8, the liver is divided into 8 segments according to the couinaud segmentation standard, such as S1-S2 section, S2-S3 section, S3-S4 section, For sections S4-S8, etc., there is also a corresponding standard cutting plan for liver resection, and one segment or multiple segments can be resected according to the actual situation of the patient. In other words, the anatomical plane or segmented section is the surgical cutting plane.

After the first section plane is determined, the first section plane can be displayed.

Through the above method, it is possible to quickly determine the first section plane according to the predetermined surgical plan or tissue and organ type, without the need for the user to temporarily manually construct the first section plane on the 3D model according to the surgical plan during the process of browsing the 3D model.

Optionally, after determining the first section plane, the method further includes:

Store the determined first section plane, and display a list containing the first section plane on the screen; receive an operation instruction from the user to select the first section plane from the list, and determine the selected one according to the operation instruction First cut plane and display.

After the first section plane is obtained through the above two methods, the obtained first section plane can also be saved, or the obtained first section plane and the corresponding section angle of view can be associated and stored, so that it can be quickly found later and display the first cut plane.

As shown in FIG. 8 , each cross-section can be displayed on the screen, and each cross-section is each first section plane for the user to click. For example, the segment names on both sides of the segmented section can be used to name each first cutting plane. When receiving an operation instruction for the user to select a first section plane from the list, the selected first section plane is determined and displayed according to the operation instruction. For example, when the user clicks on the S3-S4 section, it is determined that the first section is the segmented section between the S3 section and the S4 section, and the sectioning angle is perpendicular to the sectioned section, and the first section is displayed on the screen. A cut plane.

Fig. 9 is a schematic diagram showing a human body model provided by an embodiment of the present invention. As shown in Fig. 9, the section planes corresponding to the cross-section, sagittal plane and coronal plane of the human body can be defined in advance, and the cross-section, Sagittal and coronal planes are named and displayed in the list of the first cutting plane. For example, when the user clicks on the cross section, the first cutting plane corresponding to the cross section is displayed on the display screen.

As mentioned above, multiple first cutting planes are determined in a predefined way. The user only needs to click on the first cutting plane on the screen, and there is no need to rotate the 3D model or the cutting angle. The user operation is simple and can be quickly followed View the 3D model of the predetermined surgical plan or tissue and organ type.

Wherein, for the same 3D model, multiple first section planes can be determined only during the process of browsing the 3D model, multiple first section planes can also be predefined before browsing, or, while browsing the 3D model can also be In the process of determining multiple first section planes and predefining multiple first section planes before browsing, the present application does not limit this.

Optionally, determining at least one surgical cutting scheme corresponding to the three-dimensional model includes:

Lesions are identified according to the three-dimensional model or medical image data used to construct the three-dimensional model; and the surgical cutting plan is determined based on at least one of the shape, size and position of the lesion.

Wherein, the surgical cutting scheme may be determined according to the actual condition of the patient, that is, the lesion is determined according to the three-dimensional model or the medical image data of the three-dimensional model constructed by the user. Specifically, the three-dimensional model may be compared with the standard three-dimensional model, where the standard three-dimensional model is a three-dimensional model without lesions, and a location in the three-dimensional model inconsistent with the standard three-dimensional model is determined through comparison, and the location is determined as a lesion. Alternatively, the lesion can also be determined based on the medical image data, that is, the medical image data is compared with the standard medical image data, and the position of the lesion in the constructed three-dimensional model is determined through the data difference.

After the lesion is determined, a surgical cutting plan can be determined according to information such as the shape, position, and size of the lesion, so that the cut site includes the lesion.

The above method for determining the surgical cutting plan can automatically identify the lesion and generate a surgical cutting plan based on the information of the lesion, which can omit the process of making the surgical cutting plan for the user and improve the display efficiency of the three-dimensional model.

Optionally, the method also includes:

According to the determined first section plane, the three-dimensional model is adjusted to an orientation in which the first section plane is parallel to the screen.

After the first section plane is determined, the 3D model and the first section plane are rotated as a whole to change the viewing angle of the first section plane, so as to view the section plane of the 3D model corresponding to the first section plane more clearly.

Specifically, the first cutting plane can be adjusted to a position parallel to the screen, so that the cutting angle is the same as the viewing angle of the user viewing the screen, thereby facilitating the user to view the three-dimensional model cutting plane.

In addition, in order to view the 3D model cut plane more clearly, the 3D model M, the cutting angle O, and the cut plane P can also be rotated and/or moved as a whole, so that the 3D model M can be viewed in the process of determining the first cut plane. Adjust to a suitable orientation without changing the relationship between the 3D model M, the cutting angle O, and the cutting plane P, so as to better determine the first cutting plane, so that users can browse the cutting plane of the 3D model M from any direction.

By rotating the three-dimensional model and the first sectioning plane as a whole, after the first sectioning plane is determined, the three-dimensional model and the first sectioning plane are adjusted to an orientation suitable for the user to view, thereby improving user experience.

FIG. 10 is a schematic structural diagram of a display device provided by an embodiment of the present invention. As shown in FIG. 10, the device 100 includes:

The first determining module 1001 is configured to determine and display a first section plane based on user operations;

The second determination module 1002 is configured to determine a second section plane according to the first section plane, and display a 3D model section corresponding to the second section section; the 3D model section section is the 3D model passing through the first section section The corresponding cutting plane after cutting by the two cutting planes; the second cutting plane coincides with or is parallel to the first cutting plane;

The display module 1003 is configured to generate a two-dimensional image slice corresponding to the second slice according to the medical image data, and display the two-dimensional image slice in relation to the three-dimensional model slice.

Optionally, the display module 1003 is specifically used for:

superimposing and displaying the 2D image section on the 3D model section;

The positions of the same tissue or organ in the two-dimensional image slice and the three-dimensional model slice are respectively determined, and the two positions are set to the same color.

Optionally, the display module 1003 is specifically used for:

determining a volume box according to the maximum value in each direction in the medical image data and displaying the volume box;

determining the intersection line position between the second section plane and the volume box, and determining the boundary position of the two-dimensional image section plane in the medical image data according to the intersection line position, and determining the boundary position according to the boundary position 2D image slices.

Optionally, the display module 1003 is specifically used for:

setting the distance between the 2D image slice and the 3D model slice as a preset distance;

The corresponding tissues or organs in the 2D image section and the 3D model section are connected by a projection line.

Optionally, the device further includes: a transparent processing module, specifically for:

Receive a user's transparent processing operation instruction on the two-dimensional image section, and perform transparent processing on the two-dimensional image section according to the instruction; the transparent processing operation instruction includes a transparency percentage.

Optionally, the first determining module 1001 is specifically configured to:

After the user rotates the three-dimensional model to the orientation of interest, determine a triangular patch selected by the user on the surface of the three-dimensional model; determine the first cutting plane according to the plane where the triangular patch is located, and displaying said first cut plane;

Alternatively, display the section view angle, when the user rotates the 3D model to the orientation of interest or the user rotates the section view angle to the interest angle of view, any plane perpendicular to the section view angle is determined as first cut plane, and displays the first cut plane.

Optionally, the first determining module 1001 is specifically configured to:

Determine at least one surgical cutting plan corresponding to the three-dimensional model, and determine at least one first section plane according to the surgical cutting plan; the surgical cutting plan includes cutting planes and/or surgical positions; or, determine the three-dimensional model type, at least one first section plane is determined on the anatomical plane or section section corresponding to the type of the three-dimensional model;

Display the first cut plane.

Optionally, the first determination module 1001 is also used for:

storing the determined first cutting plane, and displaying a list containing the first cutting plane on the screen;

An operation instruction for the user to select a first section plane from the list is received, and a selected first section plane is determined and displayed according to the operation instruction.

Optionally, when the first determining module 1001 determines at least one surgical cutting plan corresponding to the three-dimensional model, it is specifically used to:

Identifying lesions according to the three-dimensional model or medical image data used to construct the three-dimensional model;

The surgical cutting scheme is determined based on at least one of information on the shape, size and location of the lesion.

Optionally, the device further includes: an adjustment module, specifically used for:

According to the determined first section plane, the three-dimensional model is adjusted to an orientation in which the first section plane is parallel to the screen.

Optionally, the second determining module 1002 is specifically configured to: determine the second section plane according to a user-triggered drag operation on the first section plane.

The display device provided by the embodiment of the present invention can implement the display method of the above embodiment as shown in FIG. 2 , and its implementation principle and technical effect are similar, and will not be repeated here.

FIG. 11 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present invention. As shown in FIG. 11 , the electronic device provided in this embodiment includes: at least one processor 1101 and a memory 1102 . Wherein, the processor 1101 and the memory 1102 are connected through a bus 1103 .

In a specific implementation process, at least one processor 1101 executes the computer-executed instructions stored in the memory 1102, so that at least one processor 1101 executes the methods in the foregoing method embodiments.

For the specific implementation process of the processor 1101, reference may be made to the foregoing method embodiments. The implementation principles and technical effects thereof are similar, and details are not repeated here in this embodiment.

In the above-mentioned embodiment shown in FIG. 11, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, referred to as: CPU), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, referred to as: DSP), application specific integrated circuit (English: Application Specific Integrated Circuit, referred to as: ASIC), etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in conjunction with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The memory may include high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory.

The bus may be an Industry Standard Architecture (Industry Standard Architecture, ISA) bus, a Peripheral Component Interconnect (Peripheral Component, PCI) bus, or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The embodiment of the present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the methods in the foregoing method embodiments are implemented.

The above-mentioned computer-readable storage medium, the above-mentioned readable storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk. Readable storage media can be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary readable storage medium is coupled to the processor such the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium may be located in application specific integrated circuits (Application Specific Integrated Circuits, ASIC for short). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit it; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the application. scope.

Claims (14)

1. A method of displaying, the method comprising:

determining and displaying a first cutting plane based on the operation of a user;

determining a second cutting plane according to the first cutting plane, and displaying a three-dimensional model cutting plane corresponding to the second cutting plane; the section of the three-dimensional model is a corresponding section of the three-dimensional model after being cut by the second cutting plane; the second cutting plane is coincident with or parallel to the first cutting plane;

and generating a two-dimensional image section corresponding to the second cutting section according to the medical image data, and displaying the two-dimensional image section at the three-dimensional model section in a correlation manner.

2. The method of claim 1, wherein the correlating the two-dimensional image slice to the three-dimensional model slice comprises:

overlapping and displaying the two-dimensional image section on the three-dimensional model section;

and respectively determining the positions of the same tissue or organ in the two-dimensional image section and the three-dimensional model section, and setting the two positions to be the same color.

3. The method of claim 1, wherein generating a two-dimensional image slice corresponding to the second resection plane from medical image data comprises:

determining a volume box according to the maximum value of each direction in the medical image data and displaying the volume box;

determining the intersection line position of the second cutting plane and the volume box, determining the boundary position of the two-dimensional image section in the medical image data according to the intersection line position, and determining the two-dimensional image section according to the boundary position.

4. The method of claim 1, wherein the associating the two-dimensional image slice with the three-dimensional model slice comprises:

setting the distance between the two-dimensional image section and the three-dimensional model section as a preset distance;

and connecting the two-dimensional image section with the corresponding tissues or organs in the three-dimensional model section through a projection line.

5. The method according to any one of claims 2-4, further comprising:

receiving a transparentizing operation instruction of a user on the two-dimensional image section, and performing transparentizing on the two-dimensional image section according to the instruction; the transparency processing operation instruction includes a transparency percentage.

6. The method of claim 1, wherein determining and displaying the first cutting plane based on the user's operation comprises:

after a user rotates the three-dimensional model to an interested position, determining a triangular patch selected by the user on the surface of the three-dimensional model; determining the first cutting plane according to the plane where the triangular patch is located, and displaying the first cutting plane;

or, displaying a sectioning angle, when a user rotates the three-dimensional model to an interested position or rotates the sectioning angle to an interested angle, determining any plane perpendicular to the sectioning angle as a first sectioning plane, and displaying the first sectioning plane.

7. The method of claim 1, wherein determining and displaying the first cutting plane based on the user's operation comprises:

determining at least one surgical cutting scheme corresponding to the three-dimensional model, and determining at least one first cutting plane according to the surgical cutting scheme; the surgical cutting plan includes a cutting plane and/or a surgical position; or determining the type of the three-dimensional model, and determining at least one first cutting plane from an anatomical plane or a segmented fracture plane corresponding to the type of the three-dimensional model;

and displaying the first cutting plane.

8. The method of claim 6 or 7, wherein after determining the first resection plane, the method further comprises:

storing the determined first cutting plane and displaying a list containing the first cutting plane on a screen;

and receiving an operation instruction of selecting a first cutting plane from the list by a user, and determining and displaying the selected first cutting plane according to the operation instruction.

9. The method of claim 7, wherein determining at least one surgical cutting plan corresponding to the three-dimensional model comprises:

identifying a lesion from the three-dimensional model or medical image data used to construct the three-dimensional model;

determining the surgical cutting plan based on at least one of information of a shape, a size, and a location of the lesion.

10. The method of claim 1, further comprising:

and adjusting the three-dimensional model to be in the direction parallel to the screen by the first cutting plane according to the determined first cutting plane.

11. The method of claim 1, wherein determining a second resection plane from the first resection plane comprises:

and determining a second cutting plane according to the dragging operation of the first cutting plane triggered by the user.

12. A display device, characterized in that the device comprises:

the first determining module is used for determining and displaying a first cutting plane based on the operation of a user; (ii) a

The second determining module is used for determining a second cutting plane according to the first cutting plane and displaying a three-dimensional model cutting plane corresponding to the second cutting plane; the three-dimensional model section is a section corresponding to the three-dimensional model after being cut by the second cutting section; the second cutting plane is coincident with or parallel to the first cutting plane;

and the display module is used for generating a two-dimensional image section corresponding to the second cutting section according to medical image data and displaying the two-dimensional image section in a correlation manner at the section of the three-dimensional model.

13. An electronic device, comprising: at least one processor and a memory;

the memory stores computer-executable instructions;

the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of any of claims 1-11.

14. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement the method of any one of claims 1 to 11.

Images