CN113313107B - Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge - Google Patents
Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge Download PDFInfo
- Publication number
- CN113313107B CN113313107B CN202110446806.3A CN202110446806A CN113313107B CN 113313107 B CN113313107 B CN 113313107B CN 202110446806 A CN202110446806 A CN 202110446806A CN 113313107 B CN113313107 B CN 113313107B
- Authority
- CN
- China
- Prior art keywords
- cable
- stayed bridge
- image
- cylindrical
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 61
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 61
- 238000001514 detection method Methods 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 238000013528 artificial neural network Methods 0.000 claims abstract description 4
- 230000003287 optical effect Effects 0.000 claims description 15
- 230000007547 defect Effects 0.000 claims description 14
- 238000012795 verification Methods 0.000 claims description 12
- 230000006378 damage Effects 0.000 claims description 7
- 238000003708 edge detection Methods 0.000 claims description 7
- 238000002372 labelling Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000012937 correction Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 6
- 239000000284 extract Substances 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000011241 protective layer Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/08—Construction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Economics (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Marketing (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Primary Health Care (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides an intelligent detection and identification method for multiple types of diseases on the cable surface of a cable-stayed bridge, which comprises the following steps: s1: acquiring apparent state image information of a cable of the cable-stayed bridge through automatic equipment; s2: removing the background in the cable image of the cable-stayed bridge based on an image preprocessing algorithm, and establishing an apparent state image information database of the cable-stayed bridge; s3: constructing a cable-stayed bridge cable multi-type disease detection model based on a neural network; s4: training a cable-stayed bridge cable multi-type disease detection model; s5: predicting cable images of the cable-stayed bridge, and determining disease types and pixel coordinate information of the cable images. The intelligent detection and identification method for the multi-type diseases on the cable surface of the cable-stayed bridge can quickly and accurately identify the multi-type diseases on the cable surface of the cable-stayed bridge, accurately position the positions of the diseases on the cable-stayed bridge by means of information provided by automatic acquisition equipment, and solve the problems of high cost, high danger, low efficiency, low precision and the like of manually detecting the apparent diseases of the cable-stayed bridge.
Description
Technical Field
The invention relates to the technical field of bridge engineering detection and computer vision, in particular to an intelligent detection and identification method for multi-type diseases on the surface of a cable-stayed bridge.
Background
The cable-stayed bridge is a novel bridge type of modern bridge and gradually becomes a main form of a large-span bridge. In China, the large-span cable-stayed bridge has become the most main form of a domestic large-scale bridge due to the characteristics of reasonable load distribution structure, good earthquake resistance and lower construction cost.
Apparent corrosion damage of cable-stayed bridge is a main cause of accident of most cable-stayed bridges, especially rust corrosion of steel wires in cable, broken wire failure and other diseases, and in the process of detecting and maintaining cable-stayed bridge, apparent detection and protection of cable is always taken as important detection content.
The conventional inspection method for the cable-stayed bridge cable comprises two steps: a manual detection method and a hanging basket detection method. The manual detection method is characterized in that the surface of the stay cable is inspected visually by bridge maintenance personnel or by carrying detection equipment so as to find and feed back the damage of the surface of the stay cable, and the method is high in subjectivity, low in detection efficiency and dangerous to some extent; as for the hanging basket detection and maintenance method, high-altitude operation is needed, so that the hanging basket detection and maintenance method is high in cost, low in efficiency, high in danger and capable of affecting normal traffic.
The team taught by Chongqing university Zhou Yi designs a cable surface defect detection system, extracts defect edges by using a canny operator, performs feature recognition on images by using a fuzzy aggregation method, and finally classifies defects based on gray scale and shape features of the images, so that attachments and defects on the cable surface cannot be distinguished. The Xu Fengyu team of the university of eastern and south has designed a set of cable surface damage detection system to extract the defect profile using an edge detection algorithm, then determine the defects based on the size of the defect area, and not classify the defects. The two sets of defect detection systems are used for detecting the surface defects of the cable based on the traditional image processing method, and the classification of the defects, the surface attachment and the distinction of the defects are required to be optimized and perfected.
In addition, the cable image acquisition of the cable-stayed bridge is carried out outdoors, and the acquired images have the conditions of exposure, shadow, unobvious difference between the background and the cable gray level and the like due to the influence of uneven illumination and complex background. In order to improve the precision of disease detection and identification, the invention pre-processes cable-stayed bridge cable images before deep learning network training and detection, segments cable-stayed bridge cable targets and backgrounds, and adjusts image contrast through histogram equalization; the cable-stayed bridge cable is a cylinder, distortion of surface texture can occur in the imaging process, and the curved surface can compress the surface area of an image during shooting, so that the detection and identification of diseases are not facilitated.
Deep learning is a method based on the characterization learning of data, combining low-level features to form more abstract high-level representation attribute categories or features to discover distributed feature representations of the data. In the research of image classification, most of the feature extraction processes are designed manually, the bottom features of the images are obtained through shallow learning, and a large 'semantic gap' exists between the bottom features and the high-level subjects of the images. The deep learning utilizes the set network structure to completely learn the hierarchical structural characteristics of the image from the training data, and can extract abstract characteristics which are more similar to the high-level semantics of the image, so that the expression of the method on the image recognition is far superior to that of the traditional method.
Noun interpretation:
mean shift algorithm: is a non-parametric statistical iterative algorithm for finding model points in a sample dataset, and is also an iterative algorithm for non-parametric kernel density estimation. The method is characterized in that sample points in an image feature space are clustered, and the sample points are converged at a zero-valued point, namely a modulus point, along the ascending direction of the probability density gradient.
Flood filling algorithm: also known as seed filling, is an algorithm that extracts a number of connected points from a region and distinguishes them from other adjacent regions.
Canny operator: the edge detection algorithm has good signal-to-noise ratio and detection accuracy. The principle is to carry out Gaussian smoothing on the input image, so as to reduce the error rate. And calculating the gradient amplitude and direction to estimate the edge intensity and direction at each point, and performing non-maximum suppression on the gradient amplitude according to the gradient direction. Finally, the edges are detected and connected by a double thresholding process.
Straight Line Segment Detector (LSD) algorithm: the method is a line segment detection algorithm, and can obtain a straight line segment detection result with sub-pixel level precision in a short time. Calculating the gradient sizes and directions of all points in the image, taking the points with small gradient direction change and adjacent points as a connected domain, judging whether the points need to be disconnected according to the rectangle degree of each domain to form a plurality of domains with larger rectangle degree, and finally, improving and screening all the generated domains, and reserving the domains meeting the conditions, namely, the final straight line detection result. The algorithm has the advantages of high detection speed, no need of parameter adjustment, and improvement of the accuracy of straight line detection by using an error control method.
Histogram equalization algorithm: the image enhancement algorithm is used for equalizing the gray level of an original image, reducing the gray level with fewer pixels in the image, and widening the gray level with more pixels in the image so that the histogram corresponding to the image is in a uniformly distributed form. Thereby increasing contrast, making the image detail clear, and realizing image space enhancement.
Cylindrical back projection: is the process of projecting a particular viewing area of the surface of a cylinder onto the tangent plane of the cylinder.
Disclosure of Invention
Based on the background situation, the invention aims to provide an intelligent detection and identification method for multi-type diseases on the cable surface of a cable-stayed bridge, which adopts automatic equipment to realize 100% nondestructive detection on the surface condition of a protective layer of the cable-stayed bridge, realizes intelligent detection and identification for multi-type diseases on the cable-stayed bridge by means of an image preprocessing algorithm and a deep learning algorithm, corrects a cylindrical cable image by a combined cylindrical surface unfolding algorithm, and projects a cylindrical curved surface to a two-dimensional plane so as to facilitate detection and identification for the surface diseases of the cable-stayed bridge; the problems of disease classification, cable surface attachment, disease distinction and the like are solved, the working efficiency and the safety are greatly improved, the maintenance cost is reduced, the method has good application prospect, and meanwhile, great economic value and social value are created.
The invention provides space geometric information and image information based on camera calibration.
An intelligent detection and identification method for multiple types of diseases on the surface of a cable-stayed bridge comprises the following steps:
s1: acquiring apparent state images of cables of the cable-stayed bridge, wherein the apparent state images of the cables of the cable-stayed bridge are used for representing complete apparent state images of 360 degrees of cables of the cable-stayed bridge; the cable-stayed bridge cable apparent state image comprises at least four images collected in different directions;
s2: removing the background in the cable image of the cable-stayed bridge based on an image preprocessing algorithm, and correcting the cylindrical cable image by adopting a cylindrical unfolding algorithm to obtain the cylindrical cable image; manually marking diseases on the cylindrical cable images, and establishing a cable apparent state image information database of the cable stayed bridge;
s3: dividing a cable apparent state image information database of the cable-stayed bridge into a training set and a verification set, inputting the training set into a neural network for training, and obtaining a cable-stayed bridge cable multi-type disease detection model through training;
s4: the cable-stayed bridge cable multi-type disease detection model is evaluated on a verification set, and the cable-stayed bridge cable multi-type disease detection model with the highest precision is selected as a final cable-stayed bridge cable multi-type disease detection model;
s5: acquiring apparent state images of cable-stayed bridge cables of the cable-stayed bridge to be detected, obtaining cylindrical cable images of the cable-stayed bridge cables to be detected according to the step S2, and inputting the cylindrical cable images of the cable-stayed bridge cables to be detected into a final cable-stayed bridge cable multi-type disease detection model to obtain the type and pixel coordinate position information of the diseases.
Further improvement, in the step S2, the defect on the cylindrical cable image includes scratch, damage, and peeling off the broken line.
The method is further improved, the background in the cable image of the cable-stayed bridge is removed based on an image preprocessing algorithm, and a cable apparent state image information database of the cable-stayed bridge is built, and specifically comprises the following steps:
s2.1: performing color dithering, scaling and Gaussian noise addition on the apparent state image information of the cable-stayed bridge acquired in the step S1 to perform data enhancement, and increasing the number of the apparent state images;
s2.2: pre-segmenting the apparent state image obtained in the step S2.1 by adopting a mean shift and flood filling method, and segmenting different areas;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator;
s2.4: detecting a straight line on the image after the edge is extracted by using a straight line segment detector algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary;
s2.6: removing the background outside the cable according to the cable boundary extracted in S2.5, and filling the background with white to obtain a cylindrical cable image;
s2.7: adjusting the brightness of the image through a histogram matching algorithm;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
s2.9: labeling the corrected cable images of the cable-stayed bridge by using labelImg software, labeling the categories of diseases and pixel coordinate position information to obtain positive samples, and searching for easily-confused false diseases to be used as negative samples, wherein the negative samples do not need to be labeled with any information;
s2.10: and forming positive and negative samples into a cable-stayed bridge cable apparent state image information database, dividing the database into a training set and a verification set according to the ratio of 9:1 by random sampling, wherein the training set is used for training a model, and the verification set is used for evaluating the quality of the model.
Further improvements, the cylindrical surface unfolding algorithm corrects the cylindrical cable image, comprising the following steps:
the cylindrical cable image is an ideal cylindrical image, and is unfolded along the direction of a generatrix, and a cylindrical surface observation area shot by a camera is projected onto a two-dimensional plane by adopting cylindrical back projection to form a two-dimensional image. About the center of the cylinder O w Establishing a three-dimensional coordinate system for the origin of coordinates along the optical center O of the camera c The direction is Z w An axis perpendicular to O c O w Is X in the horizontal direction of the plane of (2) w An axis, the vertical direction is Y w A shaft; taking any point on the surface of the cylinder imagePoint on the two-dimensional image corresponding to projection +.>Set as a radius r of a cylinder and a camera optical center O c Distance to projected two-dimensional image is f, camera optical center O c To the circle center O of the cylinder w Is g; the calculation formula for point P is as follows:
the angle theta is projected to X by the point P on the cylindrical surface target w Z w Axial plane and X w The included angle formed by the axes, d is the point P to X on the cylinder image w Z w A vertical distance of the axial plane; set point P at X w Z w The projection of the axial plane is point C, which is projected to Z w The axes being points A, Z w Shaft and O c O w On the same straight line, so CA is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the Camera optical center O c To the circle center O of the cylinder w The line of the projection is perpendicular to the projected two-dimensional plane and intersects at a point a, a is taken as an original point, the horizontal direction is taken as an X axis, and the vertical direction is taken as a Y axis to establish a two-dimensional coordinate system; the projection of the set point P' on the X-axis is point c, ca is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the According to the principle of similar triangle, on the premise that the center of a cylindrical target is positioned at the center of an image, the following results are obtained:
wherein m is the height of the cylindrical target, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, g is the center O of the cylindrical target w To camera optical center O c Is a distance of (2);
the coordinates of the two-dimensional point p' are deduced from formulas (2) (3):
the cylindrical cable image is mapped into a two-dimensional plane according to equation (4), and correction is achieved.
In a further improvement step S1, the apparent state image information of the cable-stayed bridge cable is acquired through an automatic device, the automatic acquisition device is provided with 4 cameras, and the 4 cameras shoot complete apparent state images of the cylindrical cable-stayed cable in 360 degrees.
Drawings
FIG. 1 is a flow chart of an intelligent detection and identification method for multiple types of diseases on the cable surface of a cable-stayed bridge;
FIG. 2 is a diagram showing the effect of the background removing algorithm provided by the invention; (a) an original image; (b) pre-segmenting the image; (c) edge detection images; (d) line segment detection images; (e) boundary fitting the image; (f) background cancellation and histogram equalization of the image.
FIG. 3 is a diagram of a cylindrical object imaging geometry provided by the present invention;
FIG. 4 is a diagram showing the result of the cylindrical surface correction provided by the present invention; (a) photographed cylindrical image; (b) a corrected cylindrical image;
FIG. 5 is a diagram of a model network architecture provided by the present invention;
FIG. 6 is a flow chart of model training provided by the present invention;
FIG. 7 shows the detection result of the cable-stayed bridge cable surface disease provided by the invention; (a) a scratch detection result; (b) a lesion detection result; (c) peeling detection result.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings, in which the described embodiments are only some, but not all, examples of the present invention. Other embodiments, which are apparent to those of ordinary skill in the art from consideration of the specification and practice of the invention disclosed herein, are intended to be within the scope of the invention as claimed.
As shown in FIG. 1, the intelligent detection and identification method for the multi-type diseases on the cable-stayed bridge cable surface provided by the invention comprises the following steps:
s1: acquiring apparent state image information of a cable of the cable-stayed bridge through automatic equipment;
in the task of detecting the apparent state of the cable-stayed bridge, it is very important to construct an apparent state database, and the quality of the database has great influence on the effect of a detection algorithm. The number of the manually shot and collected apparent state image data of the cable-stayed bridge cable is small, the cost is high and the cable-stayed bridge cable is incomplete, so that the invention adopts the automatic acquisition equipment to automatically acquire the apparent state image data of the cable-stayed bridge cable in all directions, the automatic acquisition equipment is provided with 4 cameras, the 4 cameras can shoot the complete apparent state image of the cylindrical cable-stayed cable in 360 degrees, and the acquired image is transmitted back to the ground storage equipment through a wireless network;
s2: removing the background in the cable image of the cable-stayed bridge based on an image preprocessing algorithm, correcting the cylindrical cable image by adopting a cylindrical unfolding algorithm, and establishing an apparent state image information database of the cable-stayed bridge;
based on the above technical solution, the step S2 specifically includes:
s2.1: and (3) carrying out color dithering, scaling, gaussian noise addition and other data enhancement on the apparent state image information acquired in the step (S1), and increasing the number of the apparent state images, wherein the variation range of the color dithering value is between [ -0.1, +0.1], the scaled size is 0.25,0.5,0.75,1.25,1.5,1.75,2.0 times of the original image, and the Gaussian noise is a random value taking 0.2 as the mean value and 0.3 as the standard deviation.
S2.2: pre-segmenting the image obtained in the step S2.1 by adopting a mean shift and flood filling method, segmenting different areas, firstly segmenting the image by adopting a mean shift algorithm, and then coloring the different segmented areas by adopting a flood filling algorithm;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator, firstly converting the pre-segmented image into a gray level image, and then extracting edges from the gray level image by using the edge detection Canny operator to obtain a binary edge extraction result image;
s2.4: detecting a straight line on the image after the edge is extracted by using a straight Line Segment Detector (LSD) algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary; the cable is a cylinder, the boundaries of the acquired cable images should be relatively parallel, the slope of the straight line detected in S2.4 is calculated, and parallel line segments are screened out. Calculating the distance between parallel line segments, and screening out the boundary of the cable according to the constraint that the distance between two sides of the cable is within a certain range, wherein the distance between the camera and the cable is fixed; according to statistics, the distance d between the two edges of the cable in this embodiment is in the range ofw is the width of the acquired image.
S2.6: removing the background outside the cable according to the cable boundary extracted in S2.5, and filling the background with white;
s2.7: the contrast of the image is adjusted through a histogram matching algorithm, and an effect diagram of the background removing algorithm is shown in fig. 2;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
the cylindrical cable image is an ideal cylindrical image, and is unfolded along the direction of a generatrix, and a cylindrical surface observation area shot by a camera is projected onto a two-dimensional plane by adopting cylindrical back projection to form a two-dimensional image. About the center of the cylinder O w Establishing a three-dimensional coordinate system for the origin of coordinates along the optical center O of the camera c The direction is Z w An axis perpendicular to O c O w Is X in the horizontal direction of the plane of (2) w An axis, the vertical direction is Y w A shaft; taking any point on the surface of the cylinder imagePoints on a two-dimensional image corresponding to projectionsSet as a radius r of a cylinder and a camera optical center O c Distance to projected two-dimensional image is f, camera optical center O c To a cylinder circleHeart O w Is g; the calculation formula for point P is as follows:
the angle theta is projected to X by the point P on the cylindrical surface target w Z w Axial plane and X w The included angle formed by the axes, d is the point P to X on the cylinder image w Z w A vertical distance of the axial plane; set point P at X w Z w The projection of the axial plane is point C, which is projected to Z w The axes being points A, Z w Shaft and O c O w On the same straight line, so CA is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the Camera optical center O c To the circle center O of the cylinder w The line of the projection is perpendicular to the projected two-dimensional plane and intersects at a point a, a is taken as an original point, the horizontal direction is taken as an X axis, and the vertical direction is taken as a Y axis to establish a two-dimensional coordinate system; the projection of the set point P' on the X-axis is point c, ca is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the According to the principle of similar triangle, on the premise that the center of a cylindrical target is positioned at the center of an image, the following results are obtained:
wherein m is the height of the cylindrical target, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, g is the center O of the cylindrical target w To camera optical center O c Is a distance of (2);
the coordinates of the two-dimensional point p' are deduced from formulas (2) (3):
the cylindrical cable image is mapped into a two-dimensional plane according to equation (4), and correction is achieved. The photographed cylindrical image is corrected by a cylindrical expansion algorithm as shown in fig. 4 (a), and the corrected cylindrical image is shown in fig. 4 (b).
S2.9: labeling the corrected cable image of the cable-stayed bridge by using labelImg software, labeling the type of the disease and the pixel coordinate position information to obtain a positive sample, specifically labeling to draw a minimum circumscribed rectangle at the disease, inputting the type of the disease, generating an xml file with the same name and image name, wherein the xml file contains the type of the disease and the pixel coordinate information, and then analyzing the xml file by a program to generate a txt file required by model training; searching confusing false diseases as negative samples, wherein the negative samples greatly increase the robustness of the algorithm, do not need to mark any information, and generate an empty txt file;
s2.10: the positive and negative samples form a cable database of the cable-stayed bridge, wherein the database contains 27070 images, 10200 negative sample images are in JPG format; dividing the database into a training set and a verification set according to the ratio of 9:1 by random sampling, wherein the training set is used for training a model, and the verification set is used for evaluating the quality of the model;
the cable-stayed bridge cable surface multi-type diseases comprise and are not limited to scratches, damages and peeling off broken lines;
s3: building a cable-stayed bridge cable multi-type disease detection model based on a neural network YOLOV 5;
fig. 5 is a diagram of a model network structure, an input image continuously extracts features through a backbone network, shallow features extract geometric texture information, deep features extract abstract semantic information, the network structure respectively connects the shallow features, middle features and deep features transversely through three branches, the deep features, the middle features and the shallow features are fused through a fusion channel, and finally three shallow features fused with other feature information are output, wherein the output result contains disease category information and bounding box pixel coordinate information.
S4: training a cable-stayed bridge cable multi-type disease detection model, and evaluating the model on a verification set to obtain a model with highest precision;
FIG. 6 is a model training flow chart, inputting training data, setting super parameters, the super parameters being set specifically as follows: the prior frame size is obtained through a k-means clustering algorithm, specifically anchors= 89,40,49,82,149,46,111,78,48,184,81,118,227,72,58,335,158,122, 105,194, 118,337, 69,601, 219,212, 161,550, 339,345;
batch size (batch) was set to 32, with a random gradient descent (SGD) algorithm for Momentum (Momentum), momentum value to 0.93, and weight decay coefficient (weight decay) to 0.0009; the initial learning rate was set to 0.013, and in the first 2000 iterations, the learning rate increased linearly from 0 to 0.013, and then the learning rate was changed in a cosine function manner in 4000 iterations each with a total number of iterations of 39000.
According to the invention, the model is trained on the image Net data set training sample, and then the trained network model parameters are initialized to the cable-stayed bridge cable multi-type disease detection model parameters, so that the network model training convergence speed and model performance can be improved.
Loading a pre-training model to perform network training, predicting disease types and surrounding frames of the disease types by extracting features, calculating loss function values, updating model parameters based on a back propagation algorithm, judging whether the model training is finished based on the evaluation precision of the model on a verification set, if the evaluation precision is converged to a certain value, finishing the training by storing the model parameters at the moment, and ending the model training, otherwise, continuing to train the model until the evaluation precision is converged.
S5: predicting cable images of the cable-stayed bridge after removing the background through image preprocessing, determining the disease type and pixel coordinate information of the cable images, removing the background of the cable images by using a preprocessing algorithm in the step S2 through an image acquisition device, correcting the cylindrical cable images through a cylindrical unfolding algorithm, sending the corrected images into a model trained in the step S4 for prediction, analyzing an output result to obtain the disease type and pixel coordinate position information;
the above-described series of detailed descriptions are merely specific illustrations of possible embodiments of the invention, which are not intended to limit the scope of the invention, and various changes made by those skilled in the art without departing from the spirit of the invention.
Claims (1)
1. The intelligent detection and identification method for the multi-type diseases on the surface of the cable-stayed bridge is characterized by comprising the following steps of:
s1: acquiring apparent state images of cables of the cable-stayed bridge, wherein the apparent state images of the cables of the cable-stayed bridge are used for representing complete apparent state images of 360 degrees of cables of the cable-stayed bridge; the cable-stayed bridge cable apparent state image comprises at least four images collected in different directions; the method comprises the steps that apparent state image information of a cable-stayed bridge cable is acquired through automatic equipment, the automatic acquisition equipment is provided with 4 cameras, and the 4 cameras shoot complete apparent state images of a cylindrical cable-stayed cable in 360 degrees;
s2: removing the background in the cable image of the cable-stayed bridge based on an image preprocessing algorithm, and correcting the cylindrical cable image by adopting a cylindrical unfolding algorithm to obtain the cylindrical cable image; manually marking diseases on the cylindrical cable images, and establishing a cable apparent state image information database of the cable stayed bridge; the defects on the cylindrical cable image include scratches, damage, and flaking off the wire
S3: dividing a cable apparent state image information database of the cable-stayed bridge into a training set and a verification set, inputting the training set into a neural network for training, and obtaining a cable-stayed bridge cable multi-type disease detection model through training; the method for creating the apparent state image information database of the cable-stayed bridge specifically comprises the following steps of:
s2.1: performing color dithering, scaling and Gaussian noise addition on the apparent state image information of the cable-stayed bridge acquired in the step S1 to perform data enhancement, and increasing the number of the apparent state images; the variation range of the color jitter value is between [ -0.1, +0.1], the scaled size is 0.25,0.5,0.75,1.25,1.5,1.75,2.0 times of the original image, and the Gaussian noise is a random value with 0.2 as the mean value and 0.3 as the standard deviation;
s2.2: pre-segmenting the apparent state image obtained in the step S2.1 by adopting a mean shift and flood filling method, and segmenting different areas;
s2.3: performing edge extraction on the pre-segmented image by using an edge detection Canny operator;
s2.4: detecting a straight line on the image after the edge is extracted by using a straight line segment detector algorithm;
s2.5: fitting a straight line by using an external rectangular boundary, and extracting a cable boundary;
s2.6: removing the background outside the cable according to the cable boundary extracted in S2.5, and filling the background with white to obtain a cylindrical cable image;
s2.7: adjusting the brightness of the image through a histogram matching algorithm;
s2.8: correcting the cylindrical cable image through a cylindrical surface unfolding algorithm;
the cylindrical surface unfolding algorithm corrects the cylindrical cable image, and the method comprises the following steps of:
the cylindrical cable image is an ideal cylindrical image, and is unfolded along the direction of a generatrix, and a cylindrical surface observation area shot by a camera is projected onto a two-dimensional plane by adopting cylindrical back projection to form a two-dimensional image; about the center of the cylinder O w Establishing a three-dimensional coordinate system for the origin of coordinates along the optical center O of the camera c The direction is Z w An axis perpendicular to O c O w Is X in the horizontal direction of the plane of (2) w An axis, the vertical direction is Y w A shaft; taking any point on the surface of the cylinder imagePoint on the two-dimensional image corresponding to projection +.>Set as a radius r of a cylinder and a camera optical center O c Distance to projected two-dimensional image is f, camera optical center O c To the circle center O of the cylinder w Is g; the calculation formula for point P is as follows:
the angle theta is projected to X by the point P on the cylindrical surface target w Z w Axial plane and X w The included angle formed by the axes, d is the point P to X on the cylinder image w Z w A vertical distance of the axial plane; set point P at X w Z w The projection of the axial plane is point C, which is projected to Z w The axes being points A, Z w Shaft and O c O w On the same straight line, so CA is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the Camera optical center O c To the circle center O of the cylinder w The line of the projection is perpendicular to the projected two-dimensional plane and intersects at a point a, a is taken as an original point, the horizontal direction is taken as an X axis, and the vertical direction is taken as a Y axis to establish a two-dimensional coordinate system; the projection of the set point P' on the X-axis is point c, ca is perpendicular to O c O w The method comprises the steps of carrying out a first treatment on the surface of the According to the principle of similar triangle, on the premise that the center of a cylindrical target is positioned at the center of an image, the following results are obtained:
wherein m is the height of the cylindrical target, h is the width of the projected two-dimensional image, w is the length of the projected two-dimensional image, g is the center O of the cylindrical target w To camera optical center O c Is a distance of (2);
the coordinates of the two-dimensional point p' are deduced from formulas (2) (3):
mapping the cylindrical cable image into a two-dimensional plane according to a formula (4) to realize correction;
s2.9: labeling the corrected cable images of the cable-stayed bridge by using labelImg software, labeling the categories of diseases and pixel coordinate position information to obtain positive samples, and searching for easily-confused false diseases to be used as negative samples, wherein the negative samples do not need to be labeled with any information; drawing a minimum circumscribed rectangle at a disease position, inputting the type of the disease, generating an xml file with the same name as the image name, wherein the xml file contains the type of the disease and pixel coordinate information, and then analyzing the xml file through a program to generate a txt file required by model training; searching a confusing false disease as a negative sample, wherein the negative sample does not need to be marked with any information, and generating an empty txt file;
s2.10: the positive and negative samples form a cable-stayed bridge cable apparent state image information database, the database is divided into a training set and a verification set according to the ratio of 9:1 by random sampling, the training set is used for training a model, and the verification set is used for evaluating the quality of the model; the multi-type diseases on the cable surface of the cable-stayed bridge comprise and are not limited to scratch, damage and peeling off;
s4: the cable-stayed bridge cable multi-type disease detection model is evaluated on a verification set, and the cable-stayed bridge cable multi-type disease detection model with the highest precision is selected as a final cable-stayed bridge cable multi-type disease detection model;
s5: acquiring apparent state images of cable-stayed bridge cables of the cable-stayed bridge to be detected, obtaining cylindrical cable images of the cable-stayed bridge cables to be detected according to the step S2, and inputting the cylindrical cable images of the cable-stayed bridge cables to be detected into a final cable-stayed bridge cable multi-type disease detection model to obtain the type and pixel coordinate position information of the diseases.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110446806.3A CN113313107B (en) | 2021-04-25 | 2021-04-25 | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110446806.3A CN113313107B (en) | 2021-04-25 | 2021-04-25 | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113313107A CN113313107A (en) | 2021-08-27 |
| CN113313107B true CN113313107B (en) | 2023-08-15 |
Family
ID=77371085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110446806.3A Active CN113313107B (en) | 2021-04-25 | 2021-04-25 | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113313107B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113960069B (en) * | 2021-10-22 | 2024-03-19 | 中铁二十二局集团第五工程有限公司 | Method for establishing cable surface morphology through laser line scanning |
| CN114280061B (en) * | 2021-12-27 | 2022-10-14 | 交通运输部公路科学研究所 | Observation method for technical conditions of cable-stayed bridge cable beam anchoring area and monitoring window |
| CN115272276A (en) * | 2022-08-12 | 2022-11-01 | 东南大学 | Suspension bridge main cable subsurface disease identification method and device based on infrared light camera shooting |
| CN118887168A (en) * | 2024-07-08 | 2024-11-01 | 广西科学院 | A visual detection and location method for bridge cable defects |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191714A (en) * | 2019-12-28 | 2020-05-22 | 浙江大学 | Intelligent identification method for bridge appearance damage diseases |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101169380B (en) * | 2007-10-31 | 2010-09-15 | 重庆大学 | Intelligent detection device for bridge cable surface damage |
| CN106856003B (en) * | 2016-12-31 | 2019-06-25 | 南京理工大学 | The expansion bearing calibration of shaft-like workpiece side surface defects detection image |
| CN107796835B (en) * | 2017-10-20 | 2021-05-25 | 北京航空航天大学 | A method and device for X-ray cylindrical three-dimensional cone beam computed tomography |
| CN111127399A (en) * | 2019-11-28 | 2020-05-08 | 东南大学 | An underwater bridge pier disease identification method based on deep learning and sonar imaging |
| CN111985499B (en) * | 2020-07-23 | 2022-11-04 | 东南大学 | High-precision bridge apparent disease identification method based on computer vision |
-
2021
- 2021-04-25 CN CN202110446806.3A patent/CN113313107B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111191714A (en) * | 2019-12-28 | 2020-05-22 | 浙江大学 | Intelligent identification method for bridge appearance damage diseases |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113313107A (en) | 2021-08-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113313107B (en) | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge | |
| US11221107B2 (en) | Method for leakage detection of underground pipeline corridor based on dynamic infrared thermal image processing | |
| CN109615611B (en) | Inspection image-based insulator self-explosion defect detection method | |
| CN113469177B (en) | Deep learning-based drainage pipeline defect detection method and system | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| Huang et al. | A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery | |
| WO2019104767A1 (en) | Fabric defect detection method based on deep convolutional neural network and visual saliency | |
| CN103048329B (en) | A kind of road surface crack detection method based on active contour model | |
| CN111814686A (en) | A vision-based transmission line identification and foreign object intrusion online detection method | |
| CN111027446B (en) | Coastline automatic extraction method of high-resolution image | |
| CN112330593A (en) | Building surface crack detection method based on deep learning network | |
| CN104851086A (en) | Image detection method for cable rope surface defect | |
| CN115222884A (en) | Space object analysis and modeling optimization method based on artificial intelligence | |
| CN115880594A (en) | Intelligent dam crack detection method based on unmanned aerial vehicle visual perception and deep learning | |
| CN113435407A (en) | Small target identification method and device for power transmission system | |
| CN113313678A (en) | Automatic sperm morphology analysis method based on multi-scale feature fusion | |
| CN116563262A (en) | Building crack detection algorithm based on multiple modes | |
| CN116758421A (en) | Remote sensing image directed target detection method based on weak supervised learning | |
| CN109584206B (en) | Synthesis method of training samples of neural network in part surface defect detection | |
| CN110348342A (en) | A kind of piping disease image partition method based on full convolutional network | |
| CN105787870A (en) | Graphic image splicing fusion system | |
| CN116740758A (en) | Bird image recognition method and system for preventing misjudgment | |
| CN109544513A (en) | A kind of steel pipe end surface defect extraction knowledge method for distinguishing | |
| CN115019163A (en) | Identification method of urban elements based on multi-source big data | |
| Dong et al. | Pixel-level intelligent segmentation and measurement method for pavement multiple damages based on mobile deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |