KR102728479B1 - Image Processing Method, Apparatus, Computing Device and Storage Medium - Google Patents

Abstract

The present disclosure provides an image processing method, an apparatus, a computing device, and a storage medium. The method of the present disclosure may include: acquiring a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein images in the first image sequence have different collection positions, collection depths, or collection angles from each other, images in the second image sequence have basically the same collection angles, and images in the second image sequence are sequentially collected at different points in time; and generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one updated image, and the at least one updated image corresponds to at least one image in the second image sequence and has a different viewing angle from the at least one image.

Description

Image Processing Method, Apparatus, Computing Device and Storage Medium

The present disclosure relates to the field of image processing, and more particularly, to an image processing method, apparatus, computing device, and storage medium.

Nowadays, doctors often use medical image sequences or image sets acquired by scanning with medical scanning devices (e.g., nuclear magnetic resonance imaging (NMRI) scanners, magnetic resonance imaging (MRI) scanners, computed tomography (CT) scanners, etc.) for medical diagnosis. After acquiring one or more original images with a medical scanning device, one or more of the original images are processed to facilitate the doctor's confirmation, thereby allowing the doctor to make an easier diagnosis by referring to the processing results.

According to one aspect of the present disclosure, an image processing method is provided, the method comprising: acquiring a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein images in the first image sequence differ from each other in at least one of a collection position, a collection depth, or a collection angle, images in the second image sequence have basically the same collection angle, and images in the second image sequence are sequentially collected at different points in time; and generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, wherein the at least one update image corresponds to at least one image in the second image sequence and has a different observation angle from the at least one image.

According to another aspect of the present disclosure, an image processing device is provided, comprising: an image sequence obtaining unit for obtaining a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein images in the first image sequence have different collection positions, collection depths, or collection angles from each other, images in the second image sequence have basically the same collection angle, and images in the second image sequence are sequentially collected at different points in time; and an image sequence generating unit for generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, wherein the at least one update image corresponds to at least one image in the second image sequence and has a different observation angle from the at least one image.

According to another aspect of the present disclosure, a computing device is provided including a memory, a processor and a computer program stored in the memory, wherein the processor is configured to execute the computer program to implement an image processing method of one or more embodiments.

According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing a computer program is provided, wherein the computer program, when executed by a processor, implements an image processing method according to one or more embodiments.

According to another aspect of the present disclosure, a computer program stored on a computer-readable storage medium including instructions, which when executed by at least one processor, implements an image processing method according to one or more embodiments.

These and other aspects of the present disclosure will become apparent and will be elucidated with reference to the embodiments described below.

Additional details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments, taken in conjunction with the drawings.
FIG. 1 is an exemplary diagram illustrating an exemplary system that can implement various methods described herein according to exemplary embodiments.
Figure 2 is a flowchart illustrating an image processing method according to an exemplary embodiment.
Figures 3a to 3d illustrate some exemplary images according to exemplary embodiments of the present disclosure.
FIG. 4 is an exemplary block diagram illustrating an image processing device according to an exemplary embodiment.
FIG. 5 is a block diagram illustrating an exemplary computer device that may be applied to an exemplary embodiment.

Unless otherwise specified in this disclosure, the terms “first,” “second,” etc., used to describe various elements are not intended to limit the positional relationship, time series relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another. In some instances, the first element and the second element may refer to the same instance of an element, but in some cases, they may refer to different instances depending on the contextual description.

The terminology used in the various exemplary descriptions herein is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, there may be one or more of the elements, provided there is no particular limitation on the number of elements. As used herein, the term “plurality” means two or more, and the term “based on …” should be interpreted as “based at least in part on …” Additionally, the terms “and/or” and “at least one of …” include any one of the listed items and all possible combinations thereof.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1 is an exemplary diagram illustrating an exemplary system (100) that may implement various methods described herein according to exemplary embodiments.

Referring to FIG. 1, the system (100) includes a client device (110), a server (120), and a network (130) that communicatively couples the client device (110) and the server (120).

The client device (110) includes a display (114) and a client application (APP) (112) that can be displayed through the display (114). The client application (112) may be an application program that requires download and installation before execution, or an applet (liteapp), which is a lightweight application. If the client application (112) is an application that requires download and installation before execution, the client application (112) may be pre-installed on the client device (110) and activated. If the client application (112) is an applet, the user (102) may directly execute the client application (112) on the client device (110) without installing the client application (112), by searching for the client application (112) (e.g., the name of the client application (112)) in the host application, or by scanning a graphic code (e.g., a barcode, a QR code, etc.) of the client application (112). In some embodiments, the client device (110) may be any type of mobile computing device, including a mobile computer, a cell phone, a wearable computing device (e.g., a head-mounted device, including a smartwatch, smartglasses, etc.), or another type of mobile device. In some embodiments, the client device (110) may alternatively be a stationary computing device, such as a desktop, a server computer, or another type of stationary computing device. In some optional embodiments, the client device (110) may be or include a medical imaging printer.

The server (120) is typically a server built by an Internet Service Provider (ISP) or an Internet Content Provider (ICP). The server (120) may represent a single server, a multi-server cluster, a distributed system, or a cloud server providing basic cloud services (e.g., cloud database, cloud computing, cloud storage, cloud communications). Although FIG. 1 shows the server (120) as communicating with only one client device (110), it will be appreciated that the server (120) may provide background services to multiple client devices simultaneously.

Examples of networks (130) include combinations of communication networks, such as a LAN, a WAN, a Personal Area Network (PAN), and/or the Internet. The network (130) may be a wired or wireless network. In some embodiments, data exchanged over the network (130) is processed using technologies and/or formats including Hyper Text Markup Language (HTML), eXtensible Markup Language (XML), and the like. Additionally, all or part of the link may be encrypted using encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), and the like. In some embodiments, custom and/or proprietary data communication technologies may be used to replace or supplement the data communication technologies mentioned above.

The system (100) may also include an image collection device (140). In some embodiments, the image collection device (140) illustrated in FIG. 1 may be a medical scanning device, including but not limited to a scanning or imaging device used in a Positron Emission Tomography (PET) system, a Positron Emission Tomography with Computerized Tomography (PET/CT) system, a Single Photon Emission Computed Tomography with Computerized Tomography (SPECT/CT) system, a Computerized Tomography (CT) system, a Medical Ultrasonograph, a Nuclear Magnetic Resonance Imaging (NMRI) system, a Magnetic Resonance Imaging (MRI) system, a Cardiac Angiograph (CA) system, a Digital Radiograph (DR) system, and the like. For example, the image collection device (140) may include a digital subtraction angiography scanner, a magnetic resonance angiography scanner, a single-photon emission computed tomography scanner, a positron emission tomography scanner, a positron computed tomography scanner, a single-photon emission computed tomography scanner, a computed tomography scanner, a medical ultrasound diagnostic device, a nuclear magnetic resonance imaging scanner, a magnetic resonance imaging scanner, a digital radiography scanner, and the like. The image collection device (140) may be connected to a server (e.g., server (120) in FIG. 1 or a separate server of an imaging system, not shown) to implement image data processing, including but not limited to scan data conversion (e.g., conversion into a medical image sequence), compression, pixel correction, 3D reconstruction, and the like.

The image collection device (140) may be connected to the client device (110), for example, via a network (130), or may be connected directly to the client device in another manner to communicate with the client device.

Optionally, the system may also include a smart computing device or computing card (150). The image capture device (140) may include or be connected (e.g., detachably connected) to such a computing card (150). As one example, the computing card (150) may implement processing of image data, including but not limited to conversion, compression, pixel correction, reconstruction, etc. As another example, the computing card (150) may implement an image processing method according to an embodiment of the present disclosure.

The system may include other parts not shown, such as a data storage unit. The data storage unit may be one or more devices of a database, a data repository, or other form for storing data, and may be a conventional database, a cloud database, a distributed database, or the like. For example, direct image data formed by the image collection device (140) or medical image sequences or three-dimensional image data acquired through image processing may be stored in the data storage unit so that subsequent servers (120) and client devices (110) can retrieve them from the data storage unit. In addition, the image collection device (140) may directly provide the image data or the medical image sequences or three-dimensional image data acquired through image processing to the server (120) or the client device (110).

A user can use a client device (110) to check collected images or videos (including preliminary image data or images that have undergone analysis processing), check analysis results, interact with collected images or analysis results, input collected commands and configuration data, etc. The client device (110) can control the collection and data processing of the image collection device by transmitting configuration data, commands, or other information to the image collection device (140).

In order to implement the purpose of the embodiment of the present disclosure, in the example of FIG. 1, the client application (112) may be an image sequence management application program that may provide various functions, such as, for example, performing storage management, indexing, sorting, and classification for collected image sequences. Correspondingly, the server (120) may be a server used together with the image sequence management application program. The server (120) may provide an image sequence management service to the client application (112) running on the client device (110) based on a user's request or a command generated according to the embodiment of the present disclosure, and may, for example, manage image sequence storage in the cloud, store a designated index (sequence type, patient identifier, human body part, collection target, collection step, collection machine, whether lesion is detected, severity, etc.), and provide an image sequence to the client device by searching according to the designated index. Alternatively, the server (120) may provide or allocate such service capabilities or storage space to the client device (110), and provide corresponding image sequence management services, etc., based on a user request or a command generated according to an embodiment of the present disclosure, by a client application (112) running on the client device (110). It should be understood that the above is only one example and that the present disclosure is not limited thereto.

FIG. 2 is a flowchart illustrating an image processing method (200) according to an exemplary embodiment. The method (200) may be executed in a client device (e.g., the client device (110) illustrated in FIG. 1), that is, the execution entity of each step of the method (200) may be the client device (110) illustrated in FIG. 1. In some embodiments, the method (200) may be executed in a server (e.g., the server (120) illustrated in FIG. 1). In some embodiments, the method (200) may be executed by a combination of a client device (e.g., the client device (110)) and a server (e.g., the server (120)).

Below, each step of the method (200) is described in detail, taking as an example the client device (110) as the executor.

Referring to FIG. 2, in step 210, a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region are acquired, wherein the images in the first image sequence have different collection positions, collection depths, or collection angles from each other, the images in the second image sequence have basically the same collection angles, and the images in the second image sequence are sequentially collected at different points in time.

In step 220, a third image sequence is generated based on the first image sequence and the second image sequence, the third image sequence including at least one update image, the at least one update image corresponding to at least one image of the second image sequence and having a different viewing angle from the at least one image.

According to these embodiments, it is possible to perform 3D reconstruction of a 2D image sequence in a time series based on a 3D image sequence.

In the field of medical image collection, there already exist several methods for generating three-dimensional image representations, such as performing three-dimensional enhancement on CT images and MR images, for example. However, although various technical methods for generating three-dimensional images based on two-dimensional images already exist, it is difficult to directly transplant these methods to a two-dimensional image sequence collected in a time series manner, since a two-dimensional image collected for a human body region changes in time series according to human breathing, vascular pulsation, etc. In contrast, according to an embodiment of the present disclosure, by using a three-dimensional image sequence collected for the same human body region as a reference, it is possible to generate a reconstruction of at least one different angle of a time-series two-dimensional image.

It will be appreciated that the images in the first image sequence differ from each other in at least one of a collection location, a collection depth, or a collection angle, which may mean that the image sequence includes images collected at different locations (e.g., different probe locations), images collected at the same location but at different imaging depths, and/or images collected at different angles, and/or combinations thereof. As a non-limiting example, the first image sequence may include one set of images collected at the same angle but at a different location or depth, and another set of images collected at essentially similar locations but at different angles, but the present disclosure is not limited thereto. It will be appreciated that this may mean that the first image sequence is capable of performing a three-dimensional representation of the first human body region. As a non-limiting example, the first image sequence may be a coronary CT angiogram (CTA), a CT venogram (CTV), a magnetic resonance angiogram (MRA), or any other image sequence capable of three-dimensional representation as understood by those skilled in the art. It can be appreciated that the first image sequence is a three-dimensional representation of the first human body region, and that the images in the first image sequence need not be three-dimensional images or have specific three-dimensional representation capabilities. In contrast, the first image sequence may include a plurality of two-dimensional images collected for the first human body region, which may include a predetermined depth or three-dimensional spatial information. For example, the first image sequence may be implemented in a three-dimensional form through post-processing, rendering, enhancement, etc., but the present disclosure is not limited thereto.

In addition, it can be understood that the images in the second image sequence have essentially the same acquisition angle, and this description may mean that the acquisition angle of the images in the second image sequence satisfies a predetermined threshold value (e.g., a shake error threshold value of the probe). In other words, the second image sequence may be a time-series two-dimensional image sequence of the first human body region, which may be used, for example, to illustrate to the user the change over time of some features in the human body region, and which generally does not have a three-dimensional expression capability before applying the method according to the embodiment of the present disclosure. In example, the images in the second image sequence may also have essentially the same acquisition position and/or acquisition depth (e.g., the same acquisition cross-section). As a non-limiting example, the second image sequence may be a digital subtraction angiography (DSA) image, although the present disclosure is not limited thereto.

It will be appreciated that throughout the text, the statements “the first image sequence is for a first human body region…” and “the second image sequence is for a first human body region…” do not imply that the first image sequence and the second image sequence have exactly the same imaging or acquisition range. The statements “the first image sequence is for a first human body region…” and “the second image sequence is for a first human body region…” do not even require that the first image sequence and the second image sequence have essentially the same imaging or acquisition range, but only that the first image sequence includes sufficient information about the topology, human tissue, human organ, lesion, etc. of the human body region to generate three-dimensional data about the specific topology, human tissue, human organ, lesion, contrast agent, etc. data in the second image sequence by reference. As another example, the first image sequence may be acquired over a certain spatial range of the first human body region, while the images in the second image sequence may be located at only one cross-section of the first human body region.

In addition, it can be understood that the “view angle” can mean the angle of view of the image, and can be expressed as the direction of the two-dimensional plane on which the image is located, the direction of the normal vector, etc. If the image is an image acquired by direct collection or an image that has only undergone plane processing, the view angle can be the same as the collection angle. In other cases, the view angle can correspond to a certain specific collection angle in space, and the view angle corresponds to identifying the collection target (e.g., the first human body part or a part thereof) at the angle, even though the image may not have been collected at that angle.

In some examples, the third image sequence may include multiple updated images. As a non-limiting example, the third image sequence may include a set of updated images, the number of images in the set of updated images being equal to the number of images in the second image sequence. In other words, in this example, the third image sequence may include a time-series image sequence that updates all images in the second image sequence. In another example, the number of updated images in the set is equal to the number of images in the second image sequence that satisfy a predetermined condition, and the predetermined condition may include, but is not limited to, images at a predetermined time interval, images satisfying a quality requirement, one image frame selected every several images, etc. As another non-limiting example, the third image sequence may include multiple sets of updated images, wherein each set of updated images corresponds to one observation angle, thereby obtaining multiple time-series sequences observed from different angles. In some examples, the third image sequence may further include, in addition to the updated one or more images, one or more sets of images, the second image sequence itself, or the processed second image sequence. It should be understood that the above is merely exemplary and the present disclosure is not limited thereto.

In some embodiments, the images in the second image sequence are contrast-assisted imaging images.

The method according to the present disclosure is particularly advantageous for contrast agent-assisted imaging image sequences. On the one hand, contrast agent imaging image sequences often require multiple images collected in time sequence to observe the movement of contrast agents in the human body, so that changes in the images over time, including the increase of contrast agents and vascular pulsation at different time points, are particularly evident. On the other hand, due to limitations of existing contrast agent imaging devices, it is often difficult to collect a sufficient number of two-dimensional images from sufficiently many different angles and positions to perform reconstruction from two-dimensional images to three-dimensional images using existing methods. For at least the above reasons, there has been no method or product for three-dimensionalizing contrast agent imaging image sequences to date. In contrast, according to embodiments of the present disclosure, a three-dimensional auxiliary image of a contrast agent imaging image can be generated.

In some embodiments, the step of generating a third image sequence based on the first image sequence and the second image sequence comprises: determining, based on the first image sequence and the second image sequence, a correspondence between at least one human organ, part or tissue in the first human body region appearing in at least one image of the first image sequence and at least one human organ, part or tissue in the first human body region appearing in at least one image of the second image sequence; and generating the third image sequence based on the correspondence, the first image sequence and the second image sequence.

According to these embodiments, it is possible to implement three-dimensional visual assistance or three-dimensional reconstruction by easily generating images of different observation angles based on a second image sequence based on a three-dimensional first image sequence based on a correspondence relationship.

It should be understood that the use of the terms “organ,” “tissue,” “body part,” etc., throughout the text, is not limited to a strict medical meaning, but means, for example, a collection of one or more organs, a collection of one or more tissues, a portion of an organ, a portion of a tissue, and is intended to include the broadest meaning that can be understood as a human body part or human body part. For example, an internal organ or a portion thereof, a lesion tissue or a portion thereof, a small segment blood vessel, a topology composed of multiple blood vessels, etc., can all be referred to as a human body part, and the present disclosure is not limited thereto.

As an example, by mapping the vessel segment area, shape, contrast agent level and other parameters in the second image sequence to the corresponding vessel segment area in the three-dimensional topology structure of the first image sequence, at least one image in the third image sequence can be generated based on the correspondence, the first image sequence and the second image sequence. For example, based on an image in the first image sequence having a different viewing angle from the corresponding image in the second image sequence, or by mapping a three-dimensional representation of the first image sequence to an image having a different viewing angle from the corresponding image in the second image sequence, the corresponding image in the third image sequence can be generated based on the topology in the two-dimensional view. As another example, the generated third image sequence can have a simpler topology than the first image sequence. For example, if there are two blood vessels intersecting in a two-dimensional plane in the second image sequence, the method of the present disclosure may include a step of distinguishing the two blood vessels in a two-dimensional plane of a corresponding image of the second image sequence using the first image sequence, thereby achieving the goal of three-dimensionalization. In such an example, it may not be necessary to generate a blood vessel topology that is completely identical to the first image sequence. Other image generation or three-dimensional reconstruction methods understood by those skilled in the art may also be used, and the present disclosure is not limited thereto. As one example, as further described below, the viewing angle of at least one updated image may be selected such that the topology structure of the corresponding image in the second image sequence is updated to compensate for intersection, overlap, occlusion, and the like present in the images.

As another example, the angle, dimension (window width), position (window position), etc. of the update image in the third image sequence can be selected based on the angle, position, CT value information, etc. of the lesion or abnormal area present in the corresponding image.

In such an example, the method of the present disclosure may include a step of processing at least one of the first image sequence and the second image sequence to identify a lesion or abnormal region therein. For example, the method of the present disclosure may include a step of processing the first image sequence to identify a lesion or abnormal region therein, and further, for example, based on a correspondence established by the two images, projecting the identified lesion or abnormal region onto a corresponding location in at least one image of the second image sequence. For another example, the method of the present disclosure may include a step of processing the second image sequence to identify a lesion or abnormal region therein. The method of the present disclosure may include a step of combining and processing the first image sequence and the second image sequence, and further, for example, performing cross-verification, combination, supplementation, etc. on lesion or abnormality data extracted from each of the two. In another example, at least one of the acquired first image sequence and the acquired second image sequence may have pre-processed or pre-identified lesion or abnormality information, and the method of the present disclosure may include a step of generating an updated image based on such information, without requiring additional processing steps.

According to one or more examples, the method of the present disclosure may include the step of generating at least one update image to better display the identified or pre-identified lesion or abnormal region. For example, the update image may be generated by selecting a viewing angle that maximizes the cross-section of the lesion or abnormal region, and the dimensions, center location of the update image may be adjusted so that the lesion or abnormal region appears in the middle of the image or at another location that can be identified by the observer. As another example, the number, resolution, etc. of the generated update images may be adjusted based on the shape, size, complexity, and severity of the lesion or abnormality information. For example, when the identified lesion has a high importance level (e.g., associated with a more severe lesion), a complex shape, and/or small dimensions, images of two or more viewing angles for the lesion region may be generated, or a higher resolution, more precise image may be generated.

According to one or more embodiments, the angle, number, resolution, etc. of the third image can be intelligently selected based on the identified lesion and other information, so that the generated third image is advantageous in better displaying information of the lesion or abnormal area.

A more specific non-limiting example according to the present disclosure will be described with reference to FIGS. 3A and 3B. As illustrated in FIG. 3A, an image (310) of a second image sequence (not illustrated) of a plurality of images is illustrated, a blood vessel (311, 312, 313, 314) is illustrated, and further, the blood vessel (313) and the blood vessel (314) have an intersection in a two-dimensional plane of the image (310).

FIG. 3b is a corresponding image (320) of a third image sequence generated based on the image (310). If the third image sequence uses a three-dimensional data format, the image (320) may also be a corresponding one two-dimensional projection of the third image sequence, and it can be understood that the present disclosure is not limited thereto. As illustrated in FIG. 3b, any image (320) of a second image sequence that may include a plurality of images is illustrated, and blood vessels (321, 322, 323, 324) that may correspond to blood vessels (311, 312, 313, 314) of FIG. 3a, respectively, are illustrated. In the drawing of FIG. 3b, blood vessels (323) and blood vessels (324) no longer intersect in the two-dimensional plane of the corresponding image (320).

As an example, a method according to an embodiment of the present disclosure determines an angle (e.g., an angle of the image (320)) at which a corresponding blood vessel no longer intersects based on a blood vessel in which an intersection exists in the image (310), and generates and displays the image (320) for a user to confirm. By way of example, a time-series image sequence (not shown) including the image (320) is generated based on the determined angle, so that the contrast agent completely represents a change in a human blood vessel at an angle at which the corresponding blood vessel does not intersect. For example, the generated image sequence may have the same time range, number, etc. as the second image. In this case, the third image sequence may mean a new two-dimensional image sequence generated, and may use the form of the two-dimensional image sequence, but may provide information of a different viewpoint of the second image sequence. In some cases, the third image sequence may provide time-series three-dimensional information, and thus may be referred to as a three-dimensional representation of the second image sequence, but the present disclosure is not limited thereto. As another example, the third image sequence can include a combination of the second image sequence and the generated new angle image sequence, wherein the third image sequence has images of at least two angles of the human body region at at least one collection point. In another example, one or more angles can be determined, one or more images (or one or more image sequences) can be generated, and the third image sequence can include the generated images or image sequences, and the present disclosure is not limited thereto.

As another example, a method according to an embodiment of the present disclosure may include a step of generating a three-dimensional image for at least one viewpoint in the second image sequence (i.e., generating a three-dimensional image for a plurality of images or each image in the second image sequence) based on the first image sequence and the second image sequence. Accordingly, a user performs observations from different angles for images of at least one viewpoint, or even all viewpoints, in the second image sequence, by operations such as dragging, rotating, etc. In this example, it can be understood that the images (310, 320) may be views shown from specific angles selected by the user, and the present disclosure is not limited thereto.

In some examples, the contrast agent concentration values can be visually presented to the user. Referring to FIG. 3c, a local image (330) of one of the collected second image sequences is illustrated, where a blood vessel (331) and a blood vessel (332) intersect. In the image (330), the contrast agent concentrations of regions (3311, 3312, 3321, 3322, 333) at different depths are presented. It can be understood that in this two-dimensional image, the contrast agent concentration of the collected intersecting region (333) is a superposition of the contrast agent concentration values of both the blood vessels (331) and the blood vessel (332). In addition, although the density of each area is indicated by shading in the drawing, the drawing is only an example, and the density of different areas can be expressed in various forms such as color, brightness, depth, gradation, numerical value, numerical range, level, etc., and the density between different areas can be the same, or the density at different locations in the same area can be different, and it can be understood that the present disclosure is not limited thereto.

Referring still to FIG. 3D , according to one or more embodiments of the present disclosure, an image or image sequence including a local image (340) may be generated by generating images of different viewing angles based on a second image sequence. The blood vessels (341) and the blood vessels (342) in the image (340) may respectively correspond to the blood vessels (331) and the blood vessels (332) in the image (330), and the angles of the image (340) cause the blood vessels (341) and the blood vessels (342) to no longer intersect. In this example, the contrast value intensities of the regions (3411, 3412, 3413, 3421, 3422, 3423) may be respectively calculated and displayed in any desired manner including but not limited to a pattern, a shade, a contrast, a color, a number, and the like. Regions (3411, 3412, 3421, 3422) correspond to regions (3311, 3312, 3321, 3322) in the second image sequence, respectively, and the density values can be obtained therefrom, for example, obtained directly, or adjusted based on changes in angle and blood vessel shape. The density of regions (3413) and (3423) can be obtained through the density of region (333), for example, determined by comprehensively subtracting the density of adjacent regions of non-intersecting portions from the density of intersecting portions, or obtained through other methods including but not limited to simulation calculations, neural network analysis, etc. It will be understood that the present disclosure is not limited thereto.

Although FIGS. 3A to 3D have described some specific examples of embodiments of the present disclosure focusing on the presence or absence of crossing blood vessels, it can be understood that the present disclosure is not limited thereto. The present disclosure can achieve the effect of visual assistance or three-dimensional reconstruction by generating at least one different angle image for a two-dimensional image time series sequence based on a three-dimensional image sequence, and this technical purpose does not necessarily require that a blood vessel crossing exists in an image among the second image sequence, and does not necessarily require that a blood vessel crossing does not exist in the generated updated image, and only needs to generate images of different observation angles so that a user (e.g., a doctor, a diagnostician) can more conveniently check the blood vessel shape, contrast agent concentration, etc. of a human body area.

In some embodiments, at least one image of the first image sequence includes one or more identified vascular segment regions, wherein the identified vascular segment regions have corresponding names; and the correspondence is a naming mapping from the identified vascular segment regions in the at least one image of the first image sequence to at least one vascular segment region in the at least one image of the second image sequence.

According to these embodiments, since the information identified in the first image sequence can be effectively utilized, the transformed visual representation form of the second image sequence can be conveniently acquired. For example, in the related art, there already exists a presentation scheme of a CTA image which can provide a name of each blood vessel. Accordingly, a three-dimensional spatial range of a blood vessel corresponding to the name can be additionally acquired, and the name is used to map the first image sequence to a second image sequence collected in time series (for example, but not limited to, contrast agent imaging). As some non-limiting examples, the image collected via CTA or MRA can include the identified blood vessel name RCA, LM, LCX, LAD, etc., and through such a name, a corresponding one or more blood vessels in the corresponding second image sequence (for example, but not limited to, DSA images) can be identified, thereby completing the mapping between image sequences of different forms.

In some embodiments, at least one image of the first image sequence includes an identified lesion region, and wherein the step of generating the third image sequence based on the correspondence, the first image sequence and the second image sequence comprises the step of generating, based on the correspondence, an image portion in at least one image of the third image sequence, corresponding to the identified lesion region.

In this embodiment, the generated third image sequence facilitates the user to view the identified lesion of the first image sequence and the time-series context of the second image sequence together. In particular, the user can view the second image sequence that changes over time in a time-series manner and in three dimensions (e.g., from at least two angles), thereby facilitating the user to view the collection results, and for example, without the need to repeatedly skip through different image sequences to view and compare them. For example, this representation helps to make a more accurate diagnosis. As an example, the identified lesion may be an identified CTA plaque, but the present disclosure is not limited thereto. For example, when the second image sequence is a contrast-assisted imaging sequence, the generated third image sequence facilitates the user to view the lesion and the contrast context together, and in particular, the user can view the time-series variation of the contrast agent from at least two angles, and can view the time-series variation of the contrast agent in combination with the lesion shape, but the present disclosure is not limited thereto.

In some embodiments, the step of determining the correspondence comprises: determining a reference image among the second image sequence; obtaining a reference two-dimensional view of the first image sequence based on the reference image; and determining the correspondence based on the reference two-dimensional view and the reference image.

According to this embodiment, one image is determined as a reference from a time-series image sequence, one two-dimensional image is determined as a reference view based on a first image sequence, and by comparing the reference image and the reference view, a correspondence relationship (e.g., a naming correspondence relationship, etc.) corresponding to blood vessels in the two image sequences can be obtained. Therefore, a mapping relationship between two image sequences of different data formats can be simply and accurately established. For example, after the mapping relationship between the reference image and the reference view is established, a mapping relationship or correspondence relationship between the reference image and other images in the second image sequence can be established.

For example, a reference two-dimensional view of the first image sequence acquired based on the reference image may be such that the reference two-dimensional view has essentially the same angle (e.g., exactly the same or a deviation angle within a threshold range) as the key image frame. The reference two-dimensional view may be any image included in the first image sequence. Alternatively, the reference two-dimensional view may be acquired by projecting three-dimensional spatial data generated based on the first image sequence onto any two-dimensional plane.

The above correspondence relationship determined based on the above reference two-dimensional view and the above reference image can be obtained based on the shape of the blood vessel among the reference two-dimensional view and the reference image. As an example, the reference image can also be referred to as a key image frame. The reference image can be determined based on one or more judgment criteria, and some examples are described below.

In some embodiments, the step of obtaining a reference two-dimensional view of the first image sequence based on the reference image comprises the step of obtaining the reference two-dimensional view based on a collection angle of the second image sequence.

According to these embodiments, it is possible to obtain a reference two-dimensional view of the first image sequence by directly obtaining or indirectly calculating a capture angle of the second image sequence. For example, for some capture devices, or under some capture parameter settings, an angle in the real world corresponding to an angle of the first image sequence in three-dimensional space is known, and/or the capture angle of the second image sequence is known. Alternatively, such capture angle can be determined based on the image data, for example, by calculating the capture angle from the image data based on the location and orientation of a specific human body organ, tissue in the image, the location of a light source, etc.

It should be understood that the step of acquiring the reference two-dimensional view may include the step of selecting an image from the first image sequence that corresponds to, is close to, or is within a certain angular range of the capture angle of the second image sequence, but the description of “acquiring the reference two-dimensional view” does not require that the reference two-dimensional view be one image from the initially collected first image sequence. As previously mentioned, in some examples, full-angle three-dimensional spatial data can be generated based on the first image sequence, and the reference two-dimensional view acquired thereby can be a two-dimensional projection of such three-dimensional spatial data at one angle.

According to some embodiments, the method comprises: obtaining a plurality of observation views having a plurality of different observation angles based on the first image sequence; and determining the reference two-dimensional view from the plurality of observation views by comparing them with the reference image.

According to these embodiments, by comparing a plurality of observation views of the first image sequence with a second image sequence (specifically, a reference view of the second image sequence), if acquisition of a capture angle of the first image sequence or the second image sequence is unnecessary or impossible, a required reference two-dimensional view can still be determined. For example, by comparing the morphology of a blood vessel or a morphology of another human tissue organ in each observation view among the plurality of observation views with the morphology of a blood vessel in the reference image, an image having the closest morphology can be determined as the reference two-dimensional view.

As previously mentioned, the step of obtaining a plurality of observation views having a plurality of different observation angles based on the first image sequence may include the step of obtaining images having a plurality of different collection angles from the first image sequence, and may also include the step of performing projection on a three-dimensional representation of the first image sequence according to the plurality of projection angles to generate a plurality of projection results, but the present disclosure is not limited thereto.

In some embodiments, the method comprises: determining a first vessel segment region among the second image sequences that satisfies a first criterion, wherein the first criterion comprises at least one of a vessel segment length, a degree of contrast agent development, and an acquisition sharpness; and determining an image frame in which the first vessel segment region is located as the reference image.

In these embodiments, a more accurate correspondence can be obtained by selecting the image with the longest blood vessel, the best phenomenon, and/or the most distinct blood vessel from the second image sequence as the reference image.

In some examples, a specific vessel in the image may be first identified, for example, a specific vessel related to the current acquisition target, the aorta, the thickest vessel in the image, the vessel where the developer first arrived, the vessel with the highest developer concentration or higher than a certain threshold, etc., and the specific vessel may be used as the “first vessel segment region”, and an image that complies with the standard may be selected as a reference image. In some other examples, among all images and all vessels, a vessel segment region that best meets the requirements of the first standard (e.g., one vessel that is the longest in all images, one vessel with the best clarity) is determined as the first vessel segment region, and a reference image is selected based on the first vessel segment region. It will be appreciated that the above is exemplary and the present disclosure is not limited thereto.

In some other examples, the first criterion may additionally or alternatively include having the largest number of blood vessels and/or having the most distinct vascular branch points, etc. In some other examples, the first criterion may include having a total concentration of the developer exceeding a certain threshold, or having a certain time elapsed since ingestion of the developer, and selecting an image in which the vascular path of the developer is the longest or the longest, as reference, but the present disclosure is not limited thereto.

In some embodiments, the step of determining a reference image among the second image sequence comprises the steps of: obtaining a first shape point among the second image sequence, wherein the first shape point corresponds to at least one of a branching point, a connecting point and a turning point of a blood vessel; and determining an image frame in which the first shape point is located as the reference image.

According to these embodiments, the selected reference image may have some desired symbolic features, such as branching and turning points. In the process of 3D reconstruction, these symbolic points are very important, because, at least compared to straight sections, shape points including relationships between blood vessels, such as branching, connection, and turning, often have certain angles and special shapes, and therefore, based on these reference points, the corresponding angles in the first image sequence can be more easily identified, and the corresponding reference 2D view can be determined, thereby accurately establishing the correspondence between the first image sequence and the second image sequence.

In some examples, such feature points can be identified by analyzing the topology in the identified image. In some examples, the identification of feature points can be focused on identifying specific branch points corresponding to coronary arteries, based on parameters such as the purpose of collection, the area of the body collected (e.g., based on the current collection target being a coronary artery). In some other examples, feature points can be identified via a pre-trained network model. In another embodiment, if the accuracy requirement is not met for identifying feature points by existing algorithms, a user (e.g., a physician) can be prompted to manually mark feature points to assist in determining the most important feature points. It will be appreciated that the present disclosure is not limited thereto.

According to some embodiments, the method of the present disclosure may further include a step of acquiring a second shape point corresponding to the first shape point among the reference two-dimensional view, wherein the step of determining the correspondence based on the reference two-dimensional view and the reference image includes a step of determining a correspondence between one or more blood vessel segment regions associated with the second shape point among the reference two-dimensional view and one or more blood vessel segment regions associated with the first shape point among the reference image.

According to these embodiments, by identifying shape points having similar constant angles or directions in the reference two-dimensional view, the correspondence between the first image sequence and the second image sequence can be established more simply and accurately.

In some embodiments, when the second image includes contrast value information, the method (200) may further include the step of generating an enhanced first image sequence, wherein the enhanced first image sequence includes contrast values based on the second image sequence. In such embodiments, different angular reconstructions can be performed on the time-series image sequence based on the three-dimensional image sequence (e.g., CTA, MRA, etc.), and enhancement and rendering can be performed on the first image sequence based on the contrast values in the second image sequence, thereby reducing the need for a user (e.g., a diagnostician) to repeatedly compare different sequences, thereby allowing the user to easily obtain more comprehensive information from the image sequence that the user currently wishes to confirm, thereby further assisting the user in diagnosis, etc.

It should be understood that although each operation is depicted in the drawings in a particular order, this should not be construed as requiring that these operations be performed in any particular order depicted or in any sequential order, nor should all of the operations depicted be performed to achieve a desired result.

Throughout this disclosure, an image sequence is image data that can be directly collected and stored or otherwise transmitted to a terminal device for use by a user. The image sequence may be processed image data after undergoing various image processing. The image sequence may also undergo other analysis processes (e.g., the presence or absence of a lesion feature or a lesion analysis process) and may further include analysis results (e.g., marking a region of interest as a circle, segmentation results of a tissue, etc.). It will be understood that the present disclosure is not limited thereto.

FIG. 4 is a schematic block diagram of an image processing device (400) according to an exemplary embodiment. The image processing device (400) may include an image sequence obtaining unit (410) and an image sequence generating unit (420). The image sequence obtaining unit (410) may be configured to obtain a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein images in the first image sequence have different collection positions, different collection depths, or different collection angles from each other, images in the second image sequence have basically the same collection angle, and images in the second image sequence are sequentially collected at different points in time. The image sequence generating unit (420) may be configured to generate a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one updated image, and the at least one updated image corresponds to at least one image in the second image sequence and has a different observation angle from the at least one image.

It should be understood that each module of the device (400) illustrated in FIG. 4 corresponds to each step of the method (200) described in FIG. 2. Accordingly, the operations, features, and advantages described for the method (200) described above are also applied to the device (400) and the modules included therein. For the sake of brevity, some operations, features, and advantages are not described redundantly herein.

According to an embodiment of the present disclosure, a computing device further includes a memory, a processor, and a computer program stored in the memory, wherein the processor is configured to execute the computer program to implement steps of an image processing method according to an embodiment of the present disclosure and variations thereof.

According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium having a computer program stored thereon is further disclosed, wherein the computer program, when executed by a processor, implements the steps of an image processing method according to an embodiment of the present disclosure and variations thereof.

According to an embodiment of the present disclosure, a computer program product including a computer program is further disclosed, wherein the computer program, when executed by a processor, implements the steps of an image processing method according to an embodiment of the present disclosure and variations thereof.

While specific functionality has been discussed above with reference to specific modules, it should be noted that the functionality of each module discussed herein may be divided into multiple modules, and/or at least some of the functionality of multiple modules may be combined into a single module. The execution of a specific module discussed herein may include the specific module itself executing the operation, or alternatively, another component or module executing the operation that the specific module calls or otherwise accesses (together with the specific module executing the operation). Thus, a specific module executing the operation may include the specific module executing the operation, and/or another module executing the operation that is called by or otherwise accessed by the specific module. For example, the multiple modules described above may in some embodiments be combined into a single module, and vice versa. The phrase “entity A initiates an operation B” as used herein may mean that entity A instructs entity A to execute operation B, but entity A does not necessarily execute the operation B on its own.

It should also be understood that the present disclosure may describe various technologies in the general context of software hardware elements or program modules. Each module described above with respect to FIG. 4 may be implemented in hardware, or in hardware combined with software and/or firmware. For example, such modules may be implemented as computer program code/instructions configured to be executed by one or more processors and stored on a computer-readable storage medium. Alternatively, such modules may be implemented as hardware logic/circuitry. For example, in some embodiments, one or more of the modules described herein may be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (including a processor (e.g., a Central Processing Unit (CPU), a microcontroller, a microprocessor, a Digital Signal Processor (DSP), etc.), memory, one or more communication interfaces, and/or one or more components of other circuitry), and may include embedded firmware for selectively executing received program code and/or performing functions.

According to one aspect of the present disclosure, a computing device is provided including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program for implementing steps of any of the method embodiments described above.

According to one aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of any of the method embodiments described above.

According to one aspect of the present disclosure, a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the method embodiments described above.

Hereinafter, exemplary examples of such computer devices, non-transitory computer-readable storage media, and computer program products are described together with FIG. 5.

FIG. 5 illustrates an exemplary configuration of a computer device (500) for implementing the method described herein. For example, the server (120) and/or client device (110) illustrated in FIG. 1 may include a similar architecture to the computer device (500). The image processing device/apparatus may be implemented in whole or at least in part by the computer device (500), or a similar device or system.

The computer device (500) may be a variety of different types of devices, such as a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and/or any other suitable computer device or computing system. Examples of the computer device (500) include, but are not limited to, a desktop computer, a server computer, a notebook or netbook computer, a mobile device (e.g., a tablet computer, a cellular or other wireless telephone (e.g., a smart phone), a notepad computer, a mobile station), a wearable device (e.g., eyeglasses, a wristwatch), an entertainment device (e.g., an entertainment appliance, a set-top box communicatively coupled to a display device, a game console), a TV or other display device, an automobile computer, and the like. Thus, the computer device (500) may range from full-resource devices with large amounts of memory and processor resources (e.g., a personal computer, a game console) to low-resource devices with limited memory and/or processing resources (e.g., a traditional set-top box, a portable game console).

A computer device (500) may include at least one processor (502), memory (504), (a plurality of) communication interfaces (506), a display device (508), other input/output (I/O) devices (510), and one or more mass storage devices (512) that may communicate with each other via a system bus (514) or other suitable connection.

The processor (502) may be a single processing unit or multiple processing units, and any processing unit may include a single or multiple computing units or multiple cores. The processor (502) may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operational instructions. In addition to other functions, the processor (502) may be configured to obtain and execute computer-readable instructions stored in memory (504), mass storage device (512), or other computer-readable medium (e.g., program code of an operating system (516), program code of an application program (518), program code of another program (520), etc.).

Memory (504) and mass storage device (512) are examples of computer-readable storage media for storing instructions executed by the processor (502) to perform each of the functions described above. For example, memory (504) may typically include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.). In addition, mass storage device (512) may typically include a hard disk drive, a solid state drive, removable media, external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network attached storage, storage area networks, etc. The memory (504) and the mass storage device (512) may both be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable instructions and processor-executable program instructions as computer program code, which may be executed by a processor (502), which is a specific machine configured to implement the operations and functions described in the examples herein.

A plurality of program modules may be stored in the mass storage device (512). These programs may include an operating system (516), one or more application programs (518), other programs (520), and program data (522), which may be loaded into the memory (504) and executed. Examples of such application programs or program modules may include, for example, program logic (e.g., computer program code or instructions) for implementing the method (200) (including any suitable steps of the method (200)) and/or other embodiments described herein.

Although illustrated as being stored in the memory (504) of the computer device (500) in FIG. 5, the modules (516, 518, 520, and 522) or portions thereof may be implemented using any form of computer-readable media accessible by the computer device (500). As used herein, “computer-readable media” includes at least two types of computer-readable media, namely computer storage media and communication media.

Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROMs, digital universal disks (DVDs) or other optical storage devices, magnetic boxes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transportable medium that can be used to store information for access by a computer device.

In contrast, a communication medium may specifically embody computer-readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transport mechanism. Computer storage media, as defined herein, does not include communication media.

The computer device (500) may further include one or more communication interfaces (506) for exchanging data with other devices, such as over a network, direct connection, etc., as described above. These communication interfaces may be one or more of any type of network interface (e.g., a NIC), a wired or wireless (e.g., an IEEE 802.11 wireless LAN (WLAN)) interface, a Wi-MAX interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth ^TM interface, a near field communication (NFC) interface, etc. The communication interface (506) may facilitate communication within a variety of network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. The communication interface (506) may also provide communication with external storage devices (not shown), such as a storage array, network attached storage, a storage area network, etc.

In some examples, a display device (508), such as a monitor, for displaying information and images to a user may be included. Other I/O devices (510) may be devices that receive various inputs from a user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input/output devices, and the like.

While the present disclosure has been particularly illustrated and described in detail in the drawings and foregoing description, it should be understood that such description is to be considered illustrative and schematic rather than restrictive, and that the disclosure is not limited to the disclosed embodiments. Variations on the disclosed embodiments can be understood and implemented by those skilled in the art, who may practice the claimed subject matter by studying the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps not listed, and the word “a” or “an” does not exclude a plurality. The mere fact that certain means are recited in different dependent claims does not imply that a combination of these means cannot be used to advantage.

Claims (16)

In the image processing method,
A step of acquiring a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein the images in the first image sequence have different collection positions, collection depths or collection angles from each other, the images in the second image sequence have basically the same collection angles, and the images in the second image sequence are sequentially collected at different points in time; and
A step of generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, wherein the at least one update image corresponds to at least one image of the second image sequence and has a different viewing angle from the at least one image;
The step of generating a third image sequence based on the first image sequence and the second image sequence is:
A step of determining a correspondence between at least one human organ, part or tissue in the first human body region appearing in at least one image of the first image sequence and at least one human organ, part or tissue in the first human body region appearing in at least one image of the second image sequence, based on the first image sequence and the second image sequence; and
A step of generating the third image sequence based on the corresponding relationship, the first image sequence and the second image sequence; comprising;
The step of determining the above correspondence is:
A step of determining a reference image among the second image sequence;
A step of obtaining a reference two-dimensional view of the first image sequence based on the reference image; and
A step of determining the correspondence based on the above-mentioned reference two-dimensional view and the above-mentioned reference image; including;
The step of obtaining a reference two-dimensional view of the first image sequence based on the above reference image comprises:
An image processing method comprising the step of obtaining the reference two-dimensional view based on the collection angle of the second image sequence.
In the first paragraph,
An image processing method wherein the image in the second image sequence is a contrast agent-assisted imaging image.
delete In the first paragraph,
At least one image of said first image sequence comprises one or more identified blood vessel segment regions, wherein said identified blood vessel segment regions have corresponding names;
An image processing method wherein the correspondence is mapped via the name from the identified blood vessel segment area in the at least one image of the first image sequence to the at least one blood vessel segment area in the at least one image of the second image sequence.
In any one of paragraphs 1, 2 and 4,
An image processing method, wherein at least one image of the first image sequence includes an identified lesion region, and wherein the step of generating the third image sequence based on the correspondence, the first image sequence and the second image sequence comprises the step of generating, based on the correspondence, an image portion corresponding to the identified lesion region in at least one image of the third image sequence.
delete delete In the first paragraph,
The step of obtaining a reference two-dimensional view of the first image sequence based on the above reference image comprises:
A step of obtaining a plurality of observation views having a plurality of different observation angles based on the first image sequence; and
An image processing method comprising: a step of determining the reference two-dimensional view from the plurality of observation views by comparing the reference image with the reference image;
In any one of paragraphs 1, 2 and 4,
The step of determining a reference image among the above second image sequence is:
determining a first vascular segment region satisfying a first criterion among the second image sequences, wherein the first criterion includes at least one of a vascular segment length, a degree of contrast development, and an acquisition sharpness; and
An image processing method comprising: a step of determining an image frame in which the first blood vessel segment area is located as the reference image;
In any one of paragraphs 1, 2 and 4,
The step of determining a reference image among the above second image sequence is:
A step of acquiring a first shape point among the second image sequence, wherein the first shape point corresponds to at least one of a branch point, a connecting point and a turning point of a blood vessel; and
An image processing method comprising the step of determining an image frame in which the first shape point is located as the reference image.
In Article 10,
An image processing method further comprising a step of acquiring a second shape point corresponding to the first shape point among the reference two-dimensional views, wherein the step of determining the correspondence based on the reference two-dimensional view and the reference image includes a step of determining a correspondence between one or more blood vessel segment regions associated with the second shape point among the reference two-dimensional view and one or more blood vessel segment regions associated with the first shape point among the reference image.
In clause 2 or 4,
An image processing method further comprising the step of generating an enhanced first image sequence, wherein the enhanced first image sequence includes contrast values based on the second image sequence.
In an image processing device,
An image sequence acquisition unit for acquiring a first image sequence corresponding to a first human body region and a second image sequence corresponding to the first human body region, wherein the images in the first image sequence have different collection positions, collection depths or collection angles from each other, the images in the second image sequence have basically the same collection angles, and the images in the second image sequence are sequentially collected at different points in time; and
An image sequence generation unit for generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence includes at least one update image, wherein the at least one update image corresponds to at least one image of the second image sequence and has a different observation angle from the at least one image;
Generating a third image sequence based on the first image sequence and the second image sequence,
Based on the first image sequence and the second image sequence, determining a correspondence between at least one human organ, part or tissue in the first human body region appearing in at least one image of the first image sequence and at least one human organ, part or tissue in the first human body region appearing in at least one image of the second image sequence;
generating the third image sequence based on the above correspondence relationship, the first image sequence and the second image sequence;
Determining the above correspondence relationship is:
Determine a reference image among the above second image sequence,
Obtaining a reference two-dimensional view of the first image sequence based on the above reference image,
Including determining the correspondence based on the above reference two-dimensional view and the above reference image,
Obtaining a reference two-dimensional view of the first image sequence based on the above reference image,
An image processing device comprising obtaining the reference two-dimensional view based on the collection angle of the second image sequence.
Comprising a memory, a processor and a computer program stored in the memory,
A computing device wherein the processor is configured to execute the computer program to implement a method according to any one of claims 1, 2 and 4.
In a non-transitory computer-readable storage medium storing a computer program,
A non-transitory computer-readable storage medium implementing the steps of a method according to any one of claims 1, 2 and 4 when the computer program is executed by a processor.
In a computer program stored in a computer-readable storage medium,
A computer program comprising instructions, wherein the instructions, when executed by at least one processor, implement the steps of a method according to any one of claims 1, 2 and 4.

Images