[go: up one dir, main page]

CN112288768A - A tracking initialization decision-making system for intestinal polyp region in colonoscopy image sequence - Google Patents

A tracking initialization decision-making system for intestinal polyp region in colonoscopy image sequence Download PDF

Info

Publication number
CN112288768A
CN112288768A CN202011224652.5A CN202011224652A CN112288768A CN 112288768 A CN112288768 A CN 112288768A CN 202011224652 A CN202011224652 A CN 202011224652A CN 112288768 A CN112288768 A CN 112288768A
Authority
CN
China
Prior art keywords
intestinal polyp
region
target
intestinal
polyp
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011224652.5A
Other languages
Chinese (zh)
Other versions
CN112288768B (en
Inventor
胡珂立
胡晓昭
彭华
赵利平
范恩
余冬华
祝汉灿
沈士根
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Shaoxing
Original Assignee
University of Shaoxing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Shaoxing filed Critical University of Shaoxing
Publication of CN112288768A publication Critical patent/CN112288768A/en
Application granted granted Critical
Publication of CN112288768B publication Critical patent/CN112288768B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30028—Colon; Small intestine
    • G06T2207/30032—Colon polyp

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，包括肠息肉区域信息获取模块、肠息肉区域判定模块、目标关联区域判定模块、目标肠息肉区域中智集建模模块、和目标肠息肉区域跟踪初始化判决模块；通过对目标肠息肉区域进行中智集建模，并计算目标肠息肉区域的中智度量同理想中智度量的交叉熵，按照交叉熵越小越可能是真实肠息肉的原则，对目标肠息肉区域进行跟踪初始判决，判断为需要实施跟踪，则加入到正在跟踪的肠息肉区域集合中处理。本发明能解决将视频目标跟踪分割算法引入肠息肉序列检测中，由于息肉区域检测分割不确定性引起的视频目标跟踪误启动的技术问题。

Figure 202011224652

The invention discloses a tracking and initialization decision-making system for intestinal polyp region in colonoscopy image sequence, which is characterized by comprising an intestinal polyp region information acquisition module, an intestinal polyp region determination module, a target associated region determination module, and a target intestinal polyp region neutrino set Modeling module, and target intestinal polyp area tracking initialization decision module; by modeling the target intestinal polyp area neutrosophic set, and calculating the cross entropy between the neutrosophic metric of the target intestinal polyp area and the ideal neutrosophic metric, according to the cross entropy According to the principle that the smaller is likely to be a real intestinal polyp, an initial judgment is made for tracking the target intestinal polyp area, and if it is determined that tracking is required, it is added to the set of intestinal polyp areas being tracked for processing. The invention can solve the technical problem of false start of video target tracking caused by the uncertainty of detection and segmentation of polyp region by introducing a video target tracking and segmentation algorithm into intestinal polyp sequence detection.

Figure 202011224652

Description

Tracking initialization decision-making system for colonoscope image sequence intestinal polyp region
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a tracking initialization decision-making system for an intestinal polyp region of a colonoscope image sequence.
Background
Colorectal cancer is the second leading death in the world, and in china, the mortality rate of colorectal cancer is also at the front. Colonoscopy is the gold standard for colorectal cancer screening and is the most effective method for early detection of colorectal polyps. In the clinic, intestinal polyp detection is mainly screened by doctors, and the intestinal polyp detection rate mainly depends on the experience of medical staff and the imaging quality of an endoscope.
The existing colorectal polyp examination technology for assisting a doctor in colonoscopy is an artificial intelligence technology based on single-frame detection segmentation, but the stability of intestinal polyp segmentation in space and time sequence in an endoscope image sequence is difficult to guarantee only by single-frame information, and polyp detection is always missed to a certain extent due to the fact that the intestinal polyp is changeable in shape and size.
The video target tracking segmentation is mainly used for solving the problem that when a starting frame target segmentation area or a target surrounding frame is given, the target is continuously and automatically tracked and segmented in a subsequent frame. The existing video target tracking segmentation algorithm has made great progress in daily scene visible light image sequences, but has not been introduced into intestinal polyp sequence detection segmentation. Because the detection and segmentation result of the intestinal polyp region has uncertainty, a video target tracking and segmentation algorithm is adopted for a colonoscope image sequence, whether the target intestinal polyp region is tracked or not is judged, and no good characteristic information is available for decision tracking initialization. Therefore, blind start-up tracking easily causes the tracking mechanism to be started up by mistake, and causes failure or error of subsequent tracking.
Disclosure of Invention
In view of the above defects or improvement requirements of the prior art, the present invention provides a system for tracking and initializing colon polyp regions in a colonoscope image sequence, which aims to solve the technical problem of false start of video target tracking caused by uncertainty of polyp region detection and segmentation when a video target tracking and segmentation algorithm is introduced into detection of colon polyp sequences.
In order to achieve the above object, according to one aspect of the present invention, there is provided a system for tracking and initializing an intestinal polyp region in a colonoscope image sequence, comprising an intestinal polyp region information obtaining module, an intestinal polyp region determining module, a target associated region determining module, a target intestinal polyp region mid-intelligence set modeling module, and a target intestinal polyp region tracking and initializing determining module;
the intestinal polyp region information acquisition module is used for acquiring the current mth frame of a colonoscope image sequence, and acquiring a candidate intestinal polyp region set of the mth frame by adopting an intestinal polyp region detection algorithm for the mth frame
Figure BDA0002763244920000021
And submitting the judgment result to an intestinal polyp region judgment module; where m is the frame number, KmThe number of candidate intestinal polyp regions detected for the mth frame, i is 1, …, Km，Sm,iThe ith candidate intestinal polyp region is a pixel point set at the corresponding position in the mth frame;
the intestinal polyp region determination module is used for determining a set of intestinal polyp regions according to the m frame candidate
Figure BDA0002763244920000022
Figure BDA0002763244920000023
Tracking intestinal polyp region set
Figure BDA0002763244920000024
And the last frame of target intestinal polyp region set
Figure BDA0002763244920000025
Judging each candidate intestinal polyp region S according to the principle that the larger the distance between the m-th frame candidate intestinal polyp region and the intestinal polyp region being tracked and the target intestinal polyp region of the previous frame is, the more likely the candidate intestinal polyp region S is a newly added target intestinal polyp regionm,iWhether the new target intestinal polyp region is newly added or not is collected, and the newly added target intestinal polyp region of the mth frame is collected to form a set
Figure BDA0002763244920000026
Submitting the data to a target associated area judgment module; wherein T ismThe number of intestinal polyp regions being tracked in the mth frame, j is 1, …, Tm，
Figure BDA0002763244920000027
For the jth intestinal polyp region being tracked; n is a radical ofm-1The number of target intestinal polyp regions in the (m-1) th frame is 1, …, Nm-1，Pm-1,kA target intestinal polyp region for the (m-1) th frame; n is Nm-1+1,…,Nm，Pm,nNewly adding a target intestinal polyp region for the mth frame;
the target associated region judgment module is used for newly adding a target intestinal polyp region set to the mth frame
Figure BDA0002763244920000028
Each newly added target intestinal polyp region P in (a)m,nSearching for an intestinal polyp region with the minimum Euclidean distance between the coordinates of the pixel points at the corresponding positions within a threshold Th 2-10 in the next frame as a target associated region P of the next framem+1,nUntil the purpose of collecting N frames continuouslySet of label associated regions { Pm,n,Pm+1,n,…,Pm+n-1,nSubmitting the data to an intelligent set modeling module in the target intestinal polyp region;
the intelligent set modeling module in the target intestinal polyp region is used for collecting the newly added target intestinal polyp region of the mth frame
Figure BDA0002763244920000031
Each newly added target intestinal polyp region P in (a)m,nSet of target associated regions { P) according to its successive N framesm,n,Pm+1,n,…,Pm+N-1,nAnd performing intelligent set modeling on the target intestinal polyp region to obtain the membership degree T of an intelligent set in each newly added target intestinal polyp regionnDegree of uncertainty InAnd degree of non-membership FnSubmitting the target intestinal polyp region to a tracking initialization judgment module;
the target intestinal polyp region tracking initialization judgment module is used for judging the membership T of the noon set of each newly added target intestinal polyp regionnDegree of uncertainty InAnd degree of non-membership FnCalculating the middle intelligence measure and the ideal middle intelligence measure DnJudging whether each newly added target intestinal polyp region is to be tracked according to the principle that the smaller the cross entropy is, the more likely it is to be a real intestinal polyp, and if the judgment is that the tracking needs to be carried out, adding the target intestinal polyp region into the intestinal polyp region set which is being tracked
Figure BDA0002763244920000032
In (1).
Preferably, in the module for modeling an intelligent set in the target intestinal polyp region, the membership T of the intelligent set in each newly added target intestinal polyp region is calculated according to the following methodnDegree of uncertainty InAnd degree of non-membership Fn：
Figure BDA0002763244920000033
Wherein the operator "| · |" represents the number of pixel points in the contained region, the function σ is the standard deviation, Pt,nIs a t frame targetIntestinal polyp region, M is the maximum value of the number of pixel points in the N regions.
Preferably, in the target intestinal polyp region tracking initialization decision module, the cross entropy D of the mesointelligence measure of each newly added target intestinal polyp region and the ideal mesointelligence measure is calculated according to the following methodnAnd judging whether each newly added target intestinal polyp region needs to be tracked or not:
Figure BDA0002763244920000041
for cross entropy DnAnd (3) judging the size:
if D isnWhen the value is less than the threshold value T and is equal to 0.8, the newly added target intestinal polyp region needs to be tracked, the newly added target intestinal polyp region is sent into a video tracking system, and the newly added target intestinal polyp region is added into a currently tracked intestinal polyp region set
Figure BDA0002763244920000042
Figure BDA0002763244920000043
Performing the following steps; otherwise the intestinal polyp region does not need to be tracked.
Preferably, in the intelligent set modeling module in the target intestinal polyp region, when the frame rate of colonoscope acquisition is low, P ist,n∩Pt+1,nIs defined as a region Pt,nAnd region Pt+1,nAnd the distance between the coordinates of the middle pixel points is within a threshold Th3 of 10.
Preferably, in the intestinal polyp region determination module, each candidate intestinal polyp region S is determined as followsm,iWhether it is a newly added target intestinal polyp region:
calculating candidate intestinal polyp region S in sequencem,iWith each intestinal polyp region being tracked
Figure BDA0002763244920000044
And each target intestinal polyp region P from the previous framem-1,kIf all distances are greater than the threshold Th1, the area is considered to beSm,iFor newly added target intestinal polyp region
Figure BDA0002763244920000045
Figure BDA0002763244920000046
Otherwise the region is not a newly added targeted intestinal polyp region.
Preferably, in the intestinal polyp region determination module, the distance between two regions is: and the minimum Euclidean distance between the pixel point coordinates of the corresponding positions of the two regions.
Preferably, the threshold Th1 is in the range of [20,100 ].
Preferably, in the target-related-region determining module, the target polyp region P is newly added at presentm,nWhen a plurality of target associated regions are provided, selecting the intestinal polyp region with the minimum Euclidean distance or the maximum intersection between the intestinal polyp region and the pixel point coordinates of the corresponding position as the target associated region P of the m + b framem+b,nWherein b is more than or equal to 1.
Preferably, in the intestinal polyp region information obtaining module, the intestinal polyp region detection algorithm is a saliency network detection algorithm or a segmentation network detection algorithm.
Preferably, in the intestinal polyp region determination module, the set of intestinal polyp regions being tracked
Figure BDA0002763244920000051
The method is obtained by calculating the position of a region of a previous frame corresponding to intestinal polyps by adopting a video target tracking segmentation algorithm in a video tracking system.
In general, compared with the prior art, the above technical solution contemplated by the present invention can achieve the following beneficial effects:
according to the invention, through carrying out centralized modeling on the target intestinal polyp region, independent uncertainty measurement is increased, fuzzy information can be better expressed, and the method has superiority in describing image uncertainty characteristics. For the intestinal polyp region, decision information is described by using the membership degree, uncertainty degree and non-membership degree of the central intelligence set, so that the initial robustness of the tracked target intestinal polyp region is effectively improved, and the missing rate and the false detection rate of polyps are reduced.
Drawings
FIG. 1 is a schematic diagram of a system for tracking and initializing intestinal polyp regions in a sequence of colonoscope images in accordance with the present invention;
fig. 2 is an embodiment of a candidate intestinal polyp region tracking initialization decision provided by the present invention.
Throughout the drawings, the same reference numerals are used to designate the same elements or structures, where 1 is the first candidate intestinal polyp region, 2 is the intestinal polyp region being tracked, 3 is the second candidate intestinal polyp region, and 4 is the colonoscope image sequence.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in FIG. 1, the invention provides a system for tracking and initializing intestinal polyp regions in a colonoscope image sequence, which comprises an intestinal polyp region information acquisition module, an intestinal polyp region judgment module, a target associated region judgment module, a target intestinal polyp region middle intelligent set modeling module and a target intestinal polyp region tracking and initializing judgment module.
The intestinal polyp region information acquisition module is used for acquiring the current mth frame of a colonoscope image sequence, and acquiring a candidate intestinal polyp region set of the mth frame by adopting an intestinal polyp region detection algorithm for the mth frame
Figure BDA0002763244920000061
And submitting the judgment result to an intestinal polyp region judgment module; where m is the frame number, KmThe number of candidate intestinal polyp regions detected for the mth frame, i is 1, …, Km，Sm,iFor the ith candidate intestinal polyp region, i.e., in the mth frameA set of pixel points corresponding to the position;
the intestinal polyp region Detection algorithm is a significant network Detection algorithm or a segmented network Detection algorithm, the significant network Detection algorithm is preferably a DSS (Deep redundant Object Detection with Short Connections) Detection algorithm, and the segmented network Detection algorithm is preferably a Segnet network (A Deep relational Encoder-Decoder Architecture for Image Segmentation) Detection algorithm.
The intestinal polyp region determination module is used for determining a set of intestinal polyp regions according to the m frame candidate
Figure BDA0002763244920000062
Figure BDA0002763244920000063
Tracking intestinal polyp region set
Figure BDA0002763244920000064
And the last frame of target intestinal polyp region set
Figure BDA0002763244920000065
Judging each candidate intestinal polyp region S according to the principle that the larger the distance between the m-th frame candidate intestinal polyp region and the intestinal polyp region being tracked and the target intestinal polyp region of the previous frame is, the more likely the candidate intestinal polyp region S is a newly added target intestinal polyp regionm,iWhether the new target intestinal polyp region is newly added or not is collected, and the newly added target intestinal polyp region of the mth frame is collected to form a set
Figure BDA0002763244920000066
Submitting the data to a target associated area judgment module; wherein T ismThe number of intestinal polyp regions being tracked in the mth frame, j is 1, …, Tm，
Figure BDA0002763244920000067
The jth intestinal polyp region being tracked, namely the pixel point set of the corresponding position in the mth frame; n is a radical ofm-1The number of target intestinal polyp regions in the (m-1) th frame is 1, …, Nm-1，Pm-1,kA target intestinal polyp region for the (m-1) th frame; n is Nm-1+1,…,Nm，Pm,nNewly adding a target intestinal polyp region for the mth frame;
judging each candidate intestinal polyp region S according to the principle that the larger the distance between the m-th frame candidate intestinal polyp region and the intestinal polyp region being tracked and the target intestinal polyp region of the previous frame is, the more possible the candidate intestinal polyp region S is a newly added target intestinal polyp regionm,iWhether the target intestinal polyp region is newly added or not is specifically as follows:
calculating candidate intestinal polyp region S in sequencem,iWith each intestinal polyp region being tracked
Figure BDA0002763244920000071
And each target intestinal polyp region P from the previous framem-1,kIf all distances are greater than the threshold Th1, the region S is considered to bem,iFor newly added target intestinal polyp region
Figure BDA0002763244920000072
Figure BDA0002763244920000073
Otherwise the region is not a newly added targeted intestinal polyp region;
the set of intestinal polyp regions being tracked
Figure BDA0002763244920000074
Calculating and acquiring the intestinal polyp from a video tracking system by adopting a video target tracking segmentation algorithm according to the region position of the corresponding intestinal polyp in the previous frame;
the video target Tracking Segmentation algorithm is preferably a SimMask network (Fast on Object Tracking and Segmentation: A unity approximation) algorithm;
when m is 1, the set of intestinal polyp regions being tracked is a null set, the set of target intestinal polyp regions of the previous frame is a null set, and then each candidate intestinal polyp region S is a null setm,iJudging as newly added target intestinal polyp region Pm,i；
The distance between the two regions is: the minimum Euclidean distance between the pixel point coordinates of the corresponding positions of the two regions;
the threshold Th1 is in the range of [20,100], and the preferred value is 50.
The target associated region judgment module is used for newly adding a target intestinal polyp region set to the mth frame
Figure BDA0002763244920000075
Each newly added target intestinal polyp region P in (a)m,nSearching for an intestinal polyp region with the minimum Euclidean distance between the coordinates of the pixel points at the corresponding positions within a threshold Th 2-10 in the next frame as a target associated region P of the next framem+1,nUntil a set of target associated regions { P } of N frames is collected continuouslym,n,Pm+1,n,…,Pm+N-1,nSubmitting the data to an intelligent set modeling module in the target intestinal polyp region;
wherein the value range of N is [5,10 ];
if the target associated region meeting the condition is not found in the next frame of the currently newly added target intestinal polyp region, the target associated region of the next frame is an empty set, namely
Figure BDA0002763244920000081
And continuing to search for a newly added target intestinal polyp region P in the m + a framem,nAn intestinal polyp region with the minimum Euclidean distance between pixel point coordinates of corresponding positions within a threshold Th 2-10 is used as a target associated region P of the m + a framem+a,nWherein a is>1；
If the current newly added target intestinal polyp region Pm,nWhen a plurality of target associated regions are provided, selecting the intestinal polyp region with the minimum Euclidean distance or the maximum intersection between the intestinal polyp region and the pixel point coordinates of the corresponding position as the target associated region P of the m + b framem+b,nWherein b is more than or equal to 1.
The intelligent set modeling module in the target intestinal polyp region is used for collecting the newly added target intestinal polyp region of the mth frame
Figure BDA0002763244920000082
Each newly added target intestinal polyp region P in (a)m,nSet of target associated regions { P) according to its successive N framesm,n,Pm+1,n,…,Pm+N-1,nAnd performing intelligent set modeling on the target intestinal polyp region to obtain the membership degree T of an intelligent set in each newly added target intestinal polyp regionnDegree of uncertainty InAnd degree of non-membership FnSubmitting the target intestinal polyp region to a tracking initialization judgment module;
performing the modeling of the middle intelligence set to obtain the membership degree T of the intelligence set in each newly added target intestinal polyp regionnDegree of uncertainty InAnd degree of non-membership FnThe method specifically comprises the following steps:
Figure BDA0002763244920000083
wherein the operator "| · |" represents the number of pixel points in the contained region, the function σ is the standard deviation, Pt,nFor the target intestinal polyp region of the t frame, M is the maximum value of the number of pixel points in the N regions;
when the frame rate of the colonoscope is low, the intersection operation result of the target associated regions of the two frames is an empty set or a set with few pixel points, Pt,n∩Pt+1,nCan be defined as a region Pt,nAnd region Pt+1,nAnd the distance between the coordinates of the middle pixel points is within a threshold Th3 of 10.
The target intestinal polyp region tracking initialization judgment module is used for judging the membership T of the noon set of each newly added target intestinal polyp regionnDegree of uncertainty InAnd degree of non-membership FnCalculating the cross entropy D of the intelligence measure and the ideal intelligence measurenJudging whether each newly added target intestinal polyp region needs to be tracked according to the principle that the smaller the cross entropy is, the more likely the target intestinal polyp region is to be a real intestinal polyp;
calculating the cross entropy D of the mesopic measure of each newly added target intestinal polyp region and the ideal mesopic measurenJudging whether each newly added target intestinal polyp region is real or not according to the principle that the smaller the cross entropy is, the more likely it is to be the real intestinal polypPerforming tracking, specifically:
Figure BDA0002763244920000091
for cross entropy DnAnd (3) judging the size:
if D isnWhen the value is less than the threshold value T and is equal to 0.8, the newly added target intestinal polyp region needs to be tracked, the newly added target intestinal polyp region is sent into a video tracking system, and the newly added target intestinal polyp region is added into a currently tracked intestinal polyp region set
Figure BDA0002763244920000092
Figure BDA0002763244920000093
Performing the following steps; otherwise the intestinal polyp region does not need to be tracked.
The following are examples:
as shown in FIG. 2, the intestinal polyp region 2 being tracked in the mth frame colonoscope image sequence 4 is calculated by using the SimMask network algorithm according to the position in the (m-1) th frame to obtain the intestinal polyp region 2 being tracked in the mth frame, which is represented as
Figure BDA0002763244920000094
A Segnet network detection algorithm is applied to the mth frame colonoscope image sequence 4 to detect a first candidate intestinal polyp region 1, which is denoted as Sm,1。
Calculating the region Sm,1And region
Figure BDA0002763244920000095
And (3) judging that the first candidate intestinal polyp region 1 is a newly added target intestinal polyp region if the minimum Euclidean distance between the pixel point coordinates of the corresponding position is greater than the threshold Th1 to be 50, wherein the newly added target intestinal polyp region is represented as Pm,1And has Pm,1＝Sm,1。
The sequence of colonoscopic images 4 detects a tracking intestinal polyp region 2 at frame m +1 as indicated by
Figure BDA0002763244920000096
The first candidate intestinal polyp region 1 is denoted Sm+1,1And a second candidate intestinal polyp region 3 is denoted Sm+1,2Separately calculating the region Sm+1,1、Sm+1,2And region Pm,1Minimum euclidean distance between pixel point coordinates of corresponding positions:
D(Sm+1,1,Pm,1)＝min{‖sr,1-pt,1‖|sr,1∈Sm+1,1,pt,1∈Pm,1}
wherein s isr,1、pt,1Are respectively the region Sm+1,1And region Pm,1Pixel point coordinates of corresponding positions;
D(Sm+1,2,Pm,1)＝min{‖sr,2-pt,1‖|sr,2∈Sm+1,2,pt,1∈Pm,1}
wherein s isr,2Is a region Sm+1,2Pixel point coordinates of corresponding positions;
D(Sm+1,1,Pm,1) Is less than the threshold Th1 of 50, it is determined that the first candidate intestinal polyp region 1 is not a newly added target intestinal polyp region;
D(Sm+1,2,Pm,1) Is greater than the threshold Th1 by 50, and the region Sm+1,2And region
Figure BDA0002763244920000101
If the minimum euclidean distance between the pixel point coordinates of the corresponding position is also greater than the threshold Th1 to 50, the second candidate intestinal polyp region 3 is determined as a newly added target intestinal polyp region, which is denoted as Pm+1,2And has Pm+1,2＝Sm+1,2。
For the first newly added target intestinal polyp region Pm,1In the m +1 Th frame, the area S with the minimum Euclidean distance between the coordinates of the pixel points at the corresponding positions within the threshold Th 2-10 is searchedm+1,1As a target associated region P of the next framem+1,1And has Pm+1,1＝Sm+1,1；
For two newly added purposesRespectively obtaining target associated regions of continuous N frames from the target polyp region 1 and the target polyp region 3, and calculating the membership degree T of the parameters of the intelligent set1And T2Degree of uncertainty I1And I2And a degree of non-membership F1And F2。
Then respectively calculating the cross entropy D of the mesopic measure of the two newly added target intestinal polyp regions and the ideal mesopic measure according to the parameters of the mesopic set1And D2，D1If the threshold value T is less than 0.8, it indicates that the intestinal polyp region 1 is a real intestinal polyp region and the tracking needs to be performed; d2If the threshold value T is greater than 0.8, it means that the intestinal polyp region 3 is a true intestinal polyp region, and it is not necessary to perform tracking.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1.一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，包括肠息肉区域信息获取模块、肠息肉区域判定模块、目标关联区域判定模块、目标肠息肉区域中智集建模模块、和目标肠息肉区域跟踪初始化判决模块；1. A tracking initialization decision-making system of colonoscopy image sequence intestinal polyp region, it is characterized in that, comprise intestinal polyp region information acquisition module, intestinal polyp region determination module, target associated region determination module, target intestinal polyp region neutronomic set modeling module, and the target intestinal polyp region tracking initialization decision module; 所述肠息肉区域信息获取模块，用于获取结肠镜图像序列的当前第m帧，对第m帧采用肠息肉区域检测算法，获得第m帧候选肠息肉区域集合
Figure FDA0002763244910000011
并提交给肠息肉区域判定模块；其中m为帧号，Km为第m帧检测到的候选肠息肉区域个数，i＝1，...，Km，Sm，i为第i个候选肠息肉区域，即在第m帧中相应位置的像素点集合；The intestinal polyp region information acquisition module is used for acquiring the current mth frame of the colonoscopy image sequence, and using an intestinal polyp region detection algorithm for the mth frame to obtain the mth frame candidate intestinal polyp region set
Figure FDA0002763244910000011
and submit it to the intestinal polyp region determination module; where m is the frame number, K m is the number of candidate intestinal polyp regions detected in the mth frame, i=1,..., Km ,Sm ,i is the ith The candidate intestinal polyp region, that is, the set of pixels at the corresponding position in the mth frame; 所述肠息肉区域判定模块，用于根据第m帧候选肠息肉区域集合
Figure FDA0002763244910000012
Figure FDA0002763244910000013
正在跟踪的肠息肉区域集合
Figure FDA0002763244910000014
以及上一帧目标肠息肉区域集合
Figure FDA0002763244910000015
按照第m帧候选肠息肉区域与正在跟踪的肠息肉区域、以及上一帧目标肠息肉区域的距离越大越可能是新增的目标肠息肉区域的原则，判断每个候选肠息肉区域Sm，i是否为新增的目标肠息肉区域，收集第m帧新增的目标肠息肉区域构成集合
Figure FDA0002763244910000016
提交到目标关联区域判定模块；其中Tm为第m帧正在跟踪的肠息肉区域个数，j＝1，...，Tm，
Figure FDA0002763244910000017
为第j个正在跟踪的肠息肉区域；Nm-1为第m-1帧目标肠息肉区域个数，k＝1，...，Nm-1，Pm-1，k为第m-1帧目标肠息肉区域；n＝Nm-1+1，...，Nm，Pm，n为第m帧新增的目标肠息肉区域；The intestinal polyp region determination module is used for the set of candidate intestinal polyp regions according to the mth frame
Figure FDA0002763244910000012
Figure FDA0002763244910000013
Collection of bowel polyp areas being tracked
Figure FDA0002763244910000014
and the set of target intestinal polyp regions in the previous frame
Figure FDA0002763244910000015
According to the principle that the larger the distance between the candidate intestinal polyp region in the mth frame and the intestinal polyp region being tracked, and the target intestinal polyp region in the previous frame, the more likely it is a new target intestinal polyp region, to determine each candidate intestinal polyp region S m, Whether i is the newly added target intestinal polyp area, collect the new target intestinal polyp area in the mth frame to form a set
Figure FDA0002763244910000016
Submit to the target associated area determination module; where T m is the number of intestinal polyp areas being tracked in the mth frame, j=1,...,T m ,
Figure FDA0002763244910000017
is the jth intestinal polyp area being tracked; N m-1 is the number of target intestinal polyp areas in the m-1th frame, k=1,...,N m-1 , P m-1, k is the mth -1 frame target intestinal polyp area; n=N m-1 +1, ..., N m , P m, n is the new target intestinal polyp area in the mth frame; 所述目标关联区域判定模块，用于对第m帧新增的目标肠息肉区域集合
Figure FDA0002763244910000018
中的每个新增的目标肠息肉区域Pm，n，在下一帧中查找与其相应位置的像素点坐标间的最小欧氏距离在门限Th2＝10以内的肠息肉区域，作为下一帧的目标关联区域Pm+1，n，直至连续收集N帧的目标关联区域集合{Pm，n，Pm+1，n，…，Pm+N-1，n}，提交给目标肠息肉区域中智集建模模块；The target associated region determination module is used for the set of target intestinal polyp regions newly added in the mth frame
Figure FDA0002763244910000018
For each newly added target intestinal polyp region P m,n in the next frame, find the intestinal polyp region with the minimum Euclidean distance between the pixel coordinates of its corresponding position within the threshold Th2=10, and use it as the next frame. The target associated area P m+1, n , until the target associated area set {P m, n , P m+1, n , . . . , P m+N-1, n } of N frames is continuously collected, and submitted to the target intestinal polyp Regional Neutrophil Set Modeling Module; 所述目标肠息肉区域中智集建模模块，用于对第m帧新增的目标肠息肉区域集合
Figure FDA0002763244910000021
中的每个新增的目标肠息肉区域Pm，n，根据其连续N帧的目标关联区域集合{Pm，n，Pm+1，n，…，Pm+N-1，n}，对其进行中智集建模，获得每个新增的目标肠息肉区域中智集的隶属度Tn、不确定性度In和非隶属度Fn，提交给目标肠息肉区域跟踪初始化判决模块；The target intestinal polyp region neutronomic set modeling module is used for the newly added target intestinal polyp region set in the mth frame
Figure FDA0002763244910000021
For each newly added target intestinal polyp region P m,n , according to its target associated region set of consecutive N frames {P m, n , P m+1, n ,..., P m+N-1, n } , perform neutrosophic set modeling on it, obtain the membership degree T n , uncertainty degree In and non -membership degree F n of each new target intestinal polyp area neutrosophic set, and submit it to the target intestinal polyp area tracking initialization judgment module; 所述目标肠息肉区域跟踪初始化判决模块，用于根据每个新增的目标肠息肉区域的中智集的隶属度Tn、不确定性度In和非隶属度Fn，计算其中智度量同理想中智度量的交叉熵Dn，按照交叉熵越小越可能是真实肠息肉的原则，判断每个新增的目标肠息肉区域是否要实施跟踪，若判断为需要实施跟踪，则将其加入正在跟踪的肠息肉区域集合
Figure FDA0002763244910000022
中。The target intestinal polyp area tracking initialization decision module is used to calculate the neutrogenous measure according to the membership degree T n , the uncertainty degree In and the non-membership degree F n of the neutrosophic set of each newly added target intestinal polyp area Same as the cross-entropy D n of the ideal neutrosophic metric, according to the principle that the smaller the cross-entropy, the more likely it is the real intestinal polyp, it is judged whether each new target intestinal polyp area needs to be tracked. Join the collection of bowel polyp areas being tracked
Figure FDA0002763244910000022
middle. 2.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述目标肠息肉区域中智集建模模块中，按照如下方法计算每个新增的目标肠息肉区域中智集的隶属度Tn、不确定性度In和非隶属度Fn：2. The system for tracking and initializing a colonoscopy image sequence bowel polyp region according to claim 1, wherein, in the neutropenic set modeling module of the target bowel polyp region, each new addition is calculated according to the following method: The membership degree T n , the uncertainty degree In and the non -membership degree F n of the neutrosophic set in the target intestinal polyp region:
Figure FDA0002763244910000023
Figure FDA0002763244910000023
 其中运算符“|·|”表示所包含区域的像素点数，函数σ为标准差，Pt，n为第t帧目标肠息肉区域，M为N个区域中像素点数的最大值。The operator "|·|" represents the number of pixels in the included area, the function σ is the standard deviation, P t, n is the target intestinal polyp area in the t-th frame, and M is the maximum number of pixels in the N areas. 3.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述目标肠息肉区域跟踪初始化判决模块中，按照如下方法计算每个新增的目标肠息肉区域的中智度量同理想中智度量的交叉熵Dn，判断每个新增的目标肠息肉区域是否要实施跟踪：3 . The system for tracking and initializing a colonoscopy image sequence intestinal polyp region according to claim 1 , wherein, in the target intestinal polyp region tracking initialization decision module, each newly added target is calculated according to the following method. 4 . The cross-entropy D n between the neutrosophic metric of the intestinal polyp area and the ideal neutrosophic metric is used to determine whether each new target intestinal polyp area needs to be tracked:
Figure FDA0002763244910000031
Figure FDA0002763244910000031
 对交叉熵Dn大小进行判断：Judge the size of the cross entropy D n : 如果Dn小于门限值T＝0.8时，则对该新增的目标肠息肉区域需要实施跟踪，将其送入视频跟踪系统，加入正在跟踪的肠息肉区域集合
Figure FDA0002763244910000032
Figure FDA0002763244910000033
中；否则该肠息肉区域不需要跟踪。If D n is less than the threshold value T=0.8, the newly added target intestinal polyp area needs to be tracked, sent to the video tracking system, and added to the set of intestinal polyp areas being tracked
Figure FDA0002763244910000032
Figure FDA0002763244910000033
Medium; otherwise this area of intestinal polyps does not require tracking. 4.如权利要求2所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述目标肠息肉区域中智集建模模块中，当结肠镜采集帧率低时，Pt，n∩Pt+1，n定义为区域Pt，n与区域Pt+1，n中像素点坐标间距离在门限Th3＝10以内的像素点集合。4 . The system for tracking and initializing the colonoscopy image sequence bowel polyp region according to claim 2 , wherein, in the neutropenic set modeling module of the target bowel polyp region, when the colonoscopy acquisition frame rate is low. 5 . , P t,n ∩P t+1,n is defined as the set of pixel points whose distance between the coordinates of pixel points in the region P t,n and the region P t+1,n is within the threshold Th3=10. 5.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述肠息肉区域判定模块中，按照如下方法判断每个候选肠息肉区域Sm，i是否为新增的目标肠息肉区域：5. The system for tracking and initializing a colonoscopy image sequence intestinal polyp region according to claim 1, wherein, in the intestinal polyp region determination module, each candidate intestinal polyp region S m is determined according to the following method, Whether i is the newly added target intestinal polyp area: 依次计算候选肠息肉区域Sm，i与每个正在跟踪的肠息肉区域
Figure FDA0002763244910000034
的距离、以及与上一帧每个目标肠息肉区域Pm-1，k的距离，如果所有距离都大于门限Th1，则认为该区域Sm，i为新增的目标肠息肉区域
Figure FDA0002763244910000035
Figure FDA0002763244910000036
否则该区域不是新增的目标肠息肉区域。Calculate the candidate intestinal polyp area S m, i in turn with each intestinal polyp area being tracked
Figure FDA0002763244910000034
and the distance from each target intestinal polyp area P m-1,k in the previous frame, if all distances are greater than the threshold Th1, the area S m,i is considered to be the newly added target intestinal polyp area
Figure FDA0002763244910000035
Figure FDA0002763244910000036
Otherwise, this area is not the newly added target intestinal polyp area. 6.如权利要求1或5所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述肠息肉区域判定模块中，两个区域的距离为：两个区域相应位置的像素点坐标间的最小欧氏距离。6. The system for tracking and initializing a colonoscopy image sequence intestinal polyp region according to claim 1 or 5, characterized in that, in the intestinal polyp region determination module, the distance between the two regions is: two regions correspond to Minimum Euclidean distance between pixel coordinates of a location. 7.如权利要求5所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述门限Th1取值范围为[20，100]。7 . The tracking initialization decision-making system for colonoscopic image sequences of intestinal polyp regions according to claim 5 , wherein the threshold Th1 has a value range of [20, 100]. 8 . 8.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述目标关联区域判定模块中，当前新增的目标肠息肉区域Pm，n的目标关联区域有多个时，选取与其相应位置的像素点坐标间的欧氏距离最小或者与其交集最大的肠息肉区域作为其第m+b帧的目标关联区域Pm+b，n，其中b≥1。8. The system for tracking and initializing a colonoscopy image sequence intestinal polyp region according to claim 1, wherein, in the target associated region determination module, the current newly added target intestinal polyp region P m, n When there are multiple target associated regions, select the intestinal polyp region with the smallest Euclidean distance or the largest intersection with its corresponding pixel coordinates as the target associated region P m+b,n of the m+bth frame, where b ≥1. 9.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述肠息肉区域信息获取模块中，肠息肉区域检测算法为显著性网络检测算法或分割网络检测算法。9. The system for tracking and initializing the colonoscopy image sequence intestinal polyp region according to claim 1, wherein, in the intestinal polyp region information acquisition module, the intestinal polyp region detection algorithm is a saliency network detection algorithm or Segmentation network detection algorithm. 10.如权利要求1所述的一种结肠镜图像序列肠息肉区域的跟踪初始化决策系统，其特征在于，所述肠息肉区域判定模块中，正在跟踪的肠息肉区域集合
Figure FDA0002763244910000041
是从视频跟踪系统中，采用视频目标跟踪分割算法，根据对应肠息肉在前一帧中的区域位置计算获取的。10 . The system for tracking and initializing the intestinal polyp region of a colonoscopy image sequence according to claim 1 , wherein, in the intestinal polyp region determination module, the intestinal polyp region being tracked is set. 11 .
Figure FDA0002763244910000041
It is obtained from the video tracking system, using the video target tracking segmentation algorithm, according to the regional position of the corresponding intestinal polyp in the previous frame.
CN202011224652.5A 2020-09-27 2020-11-05 Tracking initialization decision system for intestinal polyp region of colonoscope image sequence Active CN112288768B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011034635 2020-09-27
CN2020110346355 2020-09-27

Publications (2)

Publication Number Publication Date
CN112288768A true CN112288768A (en) 2021-01-29
CN112288768B CN112288768B (en) 2024-07-26

Family

ID=74351461

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011224652.5A Active CN112288768B (en) 2020-09-27 2020-11-05 Tracking initialization decision system for intestinal polyp region of colonoscope image sequence

Country Status (1)

Country Link
CN (1) CN112288768B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118644508A (en) * 2024-08-15 2024-09-13 顺通信息技术科技(大连)有限公司 Intelligent recognition method of colonoscopy images based on machine vision

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835178A (en) * 2015-02-02 2015-08-12 郑州轻工业学院 Low SNR(Signal to Noise Ratio) motion small target tracking and identification method
CN107256552A (en) * 2017-06-14 2017-10-17 成都康托医疗设备有限公司 Polyp image identification system and method
US20180114317A1 (en) * 2016-10-21 2018-04-26 Nantomics, Llc Digital histopathology and microdissection
CN109325128A (en) * 2018-12-03 2019-02-12 深圳大学 A method and system for tracking a maneuvering target
US20190122073A1 (en) * 2017-10-23 2019-04-25 The Charles Stark Draper Laboratory, Inc. System and method for quantifying uncertainty in reasoning about 2d and 3d spatial features with a computer machine learning architecture
CN111080585A (en) * 2019-12-04 2020-04-28 绍兴文理学院 Intelligent analysis method for detecting reflective region of colon endoscope

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835178A (en) * 2015-02-02 2015-08-12 郑州轻工业学院 Low SNR(Signal to Noise Ratio) motion small target tracking and identification method
US20180114317A1 (en) * 2016-10-21 2018-04-26 Nantomics, Llc Digital histopathology and microdissection
CN107256552A (en) * 2017-06-14 2017-10-17 成都康托医疗设备有限公司 Polyp image identification system and method
US20190122073A1 (en) * 2017-10-23 2019-04-25 The Charles Stark Draper Laboratory, Inc. System and method for quantifying uncertainty in reasoning about 2d and 3d spatial features with a computer machine learning architecture
CN109325128A (en) * 2018-12-03 2019-02-12 深圳大学 A method and system for tracking a maneuvering target
CN111080585A (en) * 2019-12-04 2020-04-28 绍兴文理学院 Intelligent analysis method for detecting reflective region of colon endoscope

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LEIMAN, DAVID A 等: "A Novel Electronic Medical Record-Based Workflow to Measure and Report Colonoscopy Quality Measures", 《CLINICAL GASTROENTEROLOGY AND HEPATOLOGY》, vol. 14, no. 3, 31 March 2016 (2016-03-31) *
杨坤: "医学虚拟内窥镜系统的研究", 《中国优秀硕士学位论文全文数据库 (工程科技Ⅱ辑)》, no. 2, 15 February 2007 (2007-02-15) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118644508A (en) * 2024-08-15 2024-09-13 顺通信息技术科技(大连)有限公司 Intelligent recognition method of colonoscopy images based on machine vision
CN118644508B (en) * 2024-08-15 2024-10-15 顺通信息技术科技(大连)有限公司 Intelligent recognition method of colonoscopy images based on machine vision

Also Published As

Publication number Publication date
CN112288768B (en) 2024-07-26

Similar Documents

Publication Publication Date Title
WO2020103676A1 (en) Image identification method and apparatus, system, and storage medium
CN111383214B (en) Real-time endoscope enteroscope polyp detection system
US20210249118A1 (en) Computer Classification of Biological Tissue
WO2022037642A1 (en) Method for detecting and classifying lesion area in clinical image
CN109389086B (en) Method and system for detecting unmanned aerial vehicle image target
Souaidi et al. A new automated polyp detection network MP-FSSD in WCE and colonoscopy images based fusion single shot multibox detector and transfer learning
WO2015141692A1 (en) Image processing device, image processing method, and image processing program
JP5784404B2 (en) Image processing apparatus, image processing method, and image processing program
Ding et al. Smart electronic gastroscope system using a cloud–edge collaborative framework
CN118172342B (en) Intracavity tumor image detection method and system
CN111862090A (en) Method and system for esophageal cancer preoperative management based on artificial intelligence
CN114581709A (en) Model training, method, apparatus and medium for recognizing objects in medical images
JP2024537198A (en) COMPUTER IMPLEMENTED SYSTEM AND METHOD FOR ANALYZING QUALITY INSPECTION OF ENDOSCOPIC PROCEDURES - Patent application
CN111080585A (en) Intelligent analysis method for detecting reflective region of colon endoscope
Li et al. Development and challenges of object detection: A survey
Gao et al. Deep endpoints focusing network under geometric constraints for end-to-end biometric measurement in fetal ultrasound images
CN112288768A (en) A tracking initialization decision-making system for intestinal polyp region in colonoscopy image sequence
Li et al. Pixel-level diabetic retinopathy lesion detection using multi-scale convolutional neural network
Gao et al. Intelligent crack damage detection system in shield tunnel using combination of retinanet and optimal adaptive selection
CN116030303B (en) Video colorectal lesion typing method based on semi-supervised twin network
CN114693912B (en) Endoscopy system, storage media and equipment with eye tracking function
CN115937163B (en) A target area extraction method and system for SPECT lung perfusion imaging
CN110598592A (en) Intelligent real-time video monitoring method suitable for nursing places
Fan et al. Positive-aware lesion detection network with cross-scale feature pyramid for OCT images
Bhattacharjee et al. Detecting laryngeal cancer lesions from endoscopy images using deep ensemble model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant