[go: up one dir, main page]

CN114266299A - Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation - Google Patents

Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation Download PDF

Info

Publication number
CN114266299A
CN114266299A CN202111540584.8A CN202111540584A CN114266299A CN 114266299 A CN114266299 A CN 114266299A CN 202111540584 A CN202111540584 A CN 202111540584A CN 114266299 A CN114266299 A CN 114266299A
Authority
CN
China
Prior art keywords
steel structure
ghost
image
bridge
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111540584.8A
Other languages
Chinese (zh)
Inventor
杨怀志
秦勇
牟宗涵
王志鹏
谢征宇
朱星盛
崔京
陈平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jiaotong University
Beijing Shanghai High Speed Railway Co Ltd
Original Assignee
Beijing Jiaotong University
Beijing Shanghai High Speed Railway Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jiaotong University, Beijing Shanghai High Speed Railway Co Ltd filed Critical Beijing Jiaotong University
Priority to CN202111540584.8A priority Critical patent/CN114266299A/en
Publication of CN114266299A publication Critical patent/CN114266299A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The application provides a method and a system for detecting defects of a steel structure of a railway bridge based on unmanned aerial vehicle operation. The method and the device utilize the unmanned aerial vehicle to carry the zoom camera, cruise and fly at the speed of 2m/s in the air 50-70m away from the railway bridge, and obtain the bridge steel structure image in the shooting period of 2 s. And then, a data set is expanded through image preprocessing, a Yolov5 algorithm is used, and a Ghost Bottleneck module is added to improve a Yolov5 model, so that parameters generated by training are further greatly reduced under the condition that the final detection precision is not changed, and the memory occupancy is reduced. The method and the device can make full use of image level information, reduce requirements of model training equipment, realize detection of targets, and guarantee detection effects of structural defects of the bridge steel.

Description

Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation
Technical Field
The application relates to a railway bridge steel structure detection technology, in particular to a railway bridge steel structure defect detection method and system based on unmanned aerial vehicle operation.
Background
The steel structures and bolts in the railway bridges are often corroded and lost due to long-term weather erosion. The disappearance or the inefficacy of steel construction and bolted connection structure can cause great influence to the stability of bridge, and the corrosion of bolt can lead to relevant equipment to dismantle the difficulty, causes very huge negative effects to the safety control in later stage.
The detection of the railway bridge bolt in China mainly depends on manual detection at present. The manual mode detection efficiency is low, the danger is large, and the number of vision blind areas is large. Especially for bolts and steel structures on the outer side of the surface of the bridge, manual short-distance detection cannot be performed, so that the condition of missed detection often occurs.
With the development of related software and hardware technologies, artificial intelligence reappears in the form of deep learning, is widely researched and applied, and appears in related reports and plans released by state hospitals for many times. At present, an unmanned aerial vehicle is gradually adopted by the organization to shoot positions which are difficult to detect manually in the bridge detection process, defect data existing in pictures are learned by using a YOLOv5 algorithm, and then technical inspection is carried out on the corrosion conditions of bridge bolts and steel structures. The mode can greatly reduce visual blind areas during manual inspection, and provides better safety guarantee for railway operation.
The current target detection algorithm for engineering objects at home and abroad is mainly carried out by three steps of defining a target area, extracting target characteristics and classifying the targets. However, the traditional learning algorithm has no universality in the implementation process, needs to manually construct a model and select features, has huge workload, is difficult to jump out of a local minimum value in the training process, has low learning speed, is easy to have the phenomena of oscillation, low convergence and the like of a loss function, and therefore, the detection and identification of the low-pass-speed low-pass-rate learning algorithm are difficult in a complex environment, and has certain limitations in engineering application.
Disclosure of Invention
The defect detection method and system for the steel structure of the railway bridge based on unmanned aerial vehicle operation are provided for overcoming the defects of the prior art, the defect detection method and system for the steel structure of the railway bridge utilize the improved YOLOv5 algorithm to detect the surface defect of the steel structure of the railway bridge, image level information can be fully utilized, memory occupation is reduced, requirements of model training equipment are reduced, detection of a target is achieved, and the detection effect of the defect of the steel structure of the railway bridge is guaranteed. The technical scheme is specifically adopted in the application.
Firstly, in order to achieve the purpose, a method for detecting structural defects of the steel of the railway bridge based on unmanned aerial vehicle operation is provided, and the method comprises the following steps: firstly, shooting a railway bridge steel structure in the air within a preset distance range from a bridge through a zoom camera carried on an unmanned aerial vehicle to obtain a plurality of bridge steel structure images; secondly, preprocessing the dry bridge steel structure image shot in the first step, expanding a data set, dividing the image in the expanded data set into a training set and a testing set according to a preset proportion, and marking the defect target in the training set by using a marking tool; thirdly, adjusting images in a training set to be uniform in size, changing a Bottleneck structure adopted in a backbone of a YOLOv5 model into a Ghost Bottleneck structure, constructing an improved YOLOv5 model by using GIoU Loss as Loss of a bounding box in the last Prediction part of the YOLOv5 model, inputting a training data set into the improved YOLOv5 model obtained by construction for training, comparing the output of the model with labeling information, and adjusting model parameters until the improved YOLOv5 model obtained by training outputs a detection result consistent with the labeling information; and fourthly, inputting the test set into the improved YOLOv5 model determined by training, detecting the steel structure defect target of the railway bridge, marking and outputting a detection result.
Optionally, the method for detecting defects of a steel structure of a railroad bridge based on unmanned aerial vehicle operation as described above, wherein the preprocessing step includes performing any one of the following processing steps or performing the following processing steps successively on the dry bridge steel structure image shot in the first step: and performing rotation processing and mirror image processing on the image, adjusting the contrast of the image, adjusting the brightness of the image, and adjusting the noise of the image.
Optionally, in the method for detecting defects of a steel structure of a railroad bridge based on unmanned aerial vehicle operation, in the first step, by using a zoom camera mounted on an unmanned aerial vehicle, when the steel structure of the railroad bridge is shot in the air within a preset distance range from the bridge, the distance from the unmanned aerial vehicle to the bridge is set to be 50-70m, the flying speed of the unmanned aerial vehicle is 2m/s during shooting, and the interval period of images shot by the unmanned aerial vehicle is 2 s.
Optionally, as for any one of the above described methods for detecting defects in a steel structure of a railroad bridge based on unmanned aerial vehicle operation, in the improved YOLOv5 model, the input end includes: the system comprises a Mosaic data enhancement module, a self-adaptive anchor frame calculation module and a self-adaptive picture scaling module; in the improved YOLOv5 model, Ghost Bottleneck and Neck respectively adopt Focus structure, CSP structure and FPN + PAN structure.
Optionally, the method for detecting defects of the steel structure of the railway bridge based on the unmanned aerial vehicle operation includes that the Ghost boltleeck structure is composed of two stacked Ghost modules, a first Ghost module is an expansion layer for increasing the number of channels, a second Ghost module is used for reducing the number of channels to match shortcut, and input and output of the two Ghost modules are connected.
OptionallyThe method for detecting the defects of the steel structure of the railway bridge based on the unmanned aerial vehicle operation comprises the following steps of recording the overlapping rate of detection results as IoU in the process of detecting the defect target of the steel structure of the railway bridge in the fourth stepresultSetting the threshold value to 0.5, when IoUresult>And 0.5, considering that the target is detected, otherwise, considering that the target is detected, marking a corresponding image according to the detected target and outputting the image.
Simultaneously, for realizing above-mentioned purpose, this application still provides a railway bridge steel structure defect detecting system based on unmanned aerial vehicle operation, and it includes: the unmanned aerial vehicle is used for cruising at the flying speed of 2m/s in the air 50-70m away from the railway bridge; the zooming camera is carried on the unmanned aerial vehicle, and shoots the steel structure of the railway bridge every 2s in the air cruising process of the unmanned aerial vehicle to obtain a plurality of images of the steel structure of the bridge; the image storage unit is used for storing the dry bridge steel structure image obtained by shooting and an expanded image obtained by preprocessing the bridge steel structure image obtained by shooting; a memory storing a computer program executable on the processor; the processor, when executing the computer program, implements the following detection process: firstly, preprocessing a dry bridge steel structure image shot by a zoom camera, expanding a data set, dividing the image in the expanded data set into a training set and a test set according to a preset proportion, and marking a defect target in the training set by using a marking tool; then, adjusting images in a training set to be uniform in size, changing a Bottleneck structure adopted in a backbone of a YOLOv5 model into a Ghost Bottleneck structure, constructing an improved YOLOv5 model by using GIoU Loss as Loss of a bounding box in the last Prediction part of the YOLOv5 model, inputting a training data set into the improved YOLOv5 model obtained by construction for training, comparing the output of the model with labeling information, and adjusting model parameters until the improved YOLOv5 model obtained by training outputs a detection result consistent with the labeling information; and finally, inputting the test set into the improved YOLOv5 model determined by training, detecting the steel structure defect target of the railway bridge, marking and outputting a detection result.
Optionally, the system for detecting structural defects of a railway bridge steel based on unmanned aerial vehicle operation as described in any one of the above, wherein in the improved YOLOv5 model obtained by the processor, the input end thereof includes: the system comprises a Mosaic data enhancement module, a self-adaptive anchor frame calculation module and a self-adaptive picture scaling module; in the improved YOLOv5 model, the Ghost Bottleneck and the Neck respectively adopt Focus structure, CSP structure and FPN + PAN structure.
Optionally, the system for detecting structural defects of a railway bridge steel based on unmanned aerial vehicle operation as described in any one of the above, wherein the Ghost bottleeck structure is composed of two stacked Ghost modules, a first Ghost module is an expansion layer for increasing the number of channels, a second Ghost module is used for reducing the number of channels to match shortcut, and input and output of the two Ghost modules are connected.
Optionally, the system for detecting structural defects of a railway bridge steel based on unmanned aerial vehicle operation as described in any one of the above, wherein the improved YOLOv5 model specifically selects a YOLOv5m structural network.
Advantageous effects
The method and the device utilize the unmanned aerial vehicle to carry the zoom camera, cruise and fly at the speed of 2m/s in the air 50-70m away from the railway bridge, and obtain the bridge steel structure image in the shooting period of 2 s. And then, a data set is expanded through image preprocessing, a Yolov5 algorithm is used, and a Ghost Bottleneck module is added to improve a Yolov5 model, so that parameters generated by training are further greatly reduced under the condition that the final detection precision is not changed, and the memory occupancy is reduced. The method and the device can make full use of image level information, reduce requirements of model training equipment, realize detection of targets, and guarantee detection effects of structural defects of the bridge steel.
The zoom camera of Zen Si H20T installed on the DJI M300 RTK unmanned aerial vehicle shoots the bridge steel structure at fixed speed and frequency, and the obtained image quality is high. The YOLOv5 algorithm is used as a one-stage target detection algorithm based on a convolutional neural network, a regression method is adopted, a complex framework in the prior art can be overcome, prediction is carried out based on information of the whole picture, the detection speed is high, and learned picture features are more universal. Because the improved algorithm based on the YOLOv5 is added with the Ghost Bottleneck module, the memory occupied by the same type of model in the training process can be further reduced under the condition of ensuring that the final detection precision is basically unchanged.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application.
Drawings
The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application and not limit the application. In the drawings:
FIG. 1 is a flowchart illustrating the overall steps of the defect detection of the railroad bridge steel structure according to the present invention;
FIG. 2 is a schematic diagram of each part of a bridge;
FIG. 3 is a schematic view of the unmanned aerial vehicle shooting the steel structure of the railway bridge in flight;
FIG. 4 is a schematic diagram of a framework for the improvement of the YOLOv5 model;
fig. 5 is a portion of raw data collected by a drone;
FIG. 6 is a diagram of the results of structural defect detection of a portion of a railroad bridge;
FIG. 7 is a schematic of target detection cross-over ratio;
FIG. 8 is a diagram illustrating the convergence of the loss curve of the YOLOv5m + Ghost Bottleneck model.
Detailed Description
In order to make the purpose and technical solutions of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings of the embodiments of the present application. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the application without any inventive step, are within the scope of protection of the application.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The term "connected" as used herein may mean either a direct connection between components or an indirect connection between components via other components.
The meaning of "up, down" in this application means that when unmanned aerial vehicle just shot the direction to the railway bridge, the direction of pointing to the bridge deck by ground is promptly for last, otherwise promptly for down, rather than the specific restriction to the device mechanism of this application.
Referring to the attached drawings 1-8, the invention provides a method for inputting an unmanned aerial vehicle bridge steel structure image into a target detection network, extracting features through a convolution network, completing the task of positioning and classifying defect coordinates, and realizing automatic detection of bridge steel structure defects, which comprises the following specific steps:
step one, carrying a camera through an unmanned aerial vehicle to obtain an image of a bridge steel structure.
The aerial shooting railway bridge steel structure with the distance from the zoom camera of Zenzhi H20T on the DJI M300 RTK unmanned aerial vehicle to the bridge being 50-70M is characterized in that the flying speed is as follows: 2m/s, and an imaging interval of 2s is set. The automatic line patrol is realized by applying an accurate repeated shooting function on the basis of first manual flight of personnel, the ground sampling interval GSD is 1.28mm, and the minimum size of an identifiable object is 26 mm.
And step two, enhancing the image of the bridge steel structure and expanding the data set.
And carrying out image processing on the structural defect data of the bridge steel, and carrying out data augmentation and data set expansion by rotating and mirroring the image, adjusting the contrast and brightness of the image, increasing noise and the like. Dividing the data into a training set and a testing set according to the proportion of 8:2, marking the defects in the training Image by using a rectangular frame by using label Image marking tool software, and generating the data containing the Image name, the defect coordinates and the defect type.
And step three, improving the network structure of the YOLOv5 algorithm.
The images in the training set are resized to a uniform size, e.g., the training images may be uniformly resized to 640 x 640 pixels.
Then changing the Bottleneck structure adopted in the Yolov5 model backbone into a Ghost Bottleneck structure, constructing an improved Yolov5 model by using GIoU Loss as the Loss of a bounding box in the last Prediction part of the Yolov5 model, inputting a training data set into the improved Yolov5 model obtained by construction for training, comparing the model output with labeling information, and adjusting model parameters until the improved Yolov5 model obtained by training outputs a detection result consistent with the labeling information.
The YOLOv5 model structure is divided into four structure networks of YOLOv5s, YOLOv5m, YOLOv5l and YOLOv5x according to the difference of depth and width. The input end of the YOLOv5 network structure model comprises modules of Mosaic data enhancement, self-adaptive anchor frame calculation, self-adaptive picture scaling and the like, the Mosaic data enhancement increases the data volume and improves the precision of small target detection, and the self-adaptive anchor frame calculation and the self-adaptive picture scaling reduce the calculation amount during reasoning and increase the detection speed. The Backbone and the Neck respectively adopt a Focus structure, a CSP structure and an FPN + PAN structure to enhance the characteristic fusion capability of the network. However, the original bottleeck structure in the backbone only performs dimensional output, and the Ghost bottleeck can reduce parameters generated in the training process, reduce the occupancy rate of the GPU, reduce the lower limit of the training device, and ensure that the precision of the training result is not affected, so the original bottleeck is replaced with the Ghost bottleeck. The added Ghost Bottleneck module of the application reveals intrinsic feature information by utilizing similar feature mapping in a convolutional neural network. The Ghost Bottleneck mainly comprises two stacked Ghost modules, wherein the first Ghost module is an expansion layer for increasing the number of channels, the second Ghost module is used for reducing the number of the channels to match shortcut, and finally the input and the output of the two Ghost modules are connected.
The YOLO V5 last Prediction part uses GIoU Loss as the Loss of the bounding Box, unlike IoU which only focuses on the overlapping region, GIoU focuses on not only the overlapping region but also other non-overlapping regions, and the coincidence ratio of the two regions can be better reflected, and Box is presumed to be the mean value of the GIoU Loss function, and the smaller the Box is, the more accurate the Box is.
And step four, realizing the target detection of the defects of the bridge steel structure component based on the improved YOLOv5 algorithm.
The improved target detection algorithm based on the YOLOv5 model is used for realizing the target identification of the structural defects of the bridge steel, and the overlapping rate of the results is recorded to be IoUresultSetting the threshold value to 0.5, when IoUresult>0.5, the target is detected, and the types of the prediction frames are obtained as a true case (TP), a false positive case (FP), a true counter case (TN), a false counter case (FN), a calculation model performance evaluation index, P (accuracy), R (recall), AP (precision mean), mAP (mean precision mean).
The accuracy, recall, AP value was calculated as follows
Figure BDA0003413943680000101
Figure BDA0003413943680000102
Figure BDA0003413943680000103
In the embodiment, the unmanned aerial vehicle bridge steel structure image data set of the Jinghai railway Jinan yellow river bridge is adopted for training and detecting. The number of the bridge steel structure images shot by the unmanned aerial vehicle on the bridge of the Jinshanu railway Jinan yellow river is 500. The data size is expanded into 2000 sheets by the traditional image processing means, and the 2000 sheets are divided into a training set and a testing set according to the ratio of 8: 2. The original image is adjusted to 640 × 640 size to reduce the memory occupied by the model. And classifying the detection types into normal bolts, bolt loss, bolt corrosion and steel structure corrosion according to the detected parts.
The test compares different types of test precision, training parameters and memory occupancy before and after the improvement of the model with different sizes of YOLOv5 in the structural data set of the railway bridge steel, the detection index is mAP0.5, and when the epoch is 100, the test result is as follows:
TABLE 1 railroad bridge Steel Structure data set comparison results
Figure BDA0003413943680000104
In general, under the condition of ensuring that the precision of the test data is basically unchanged, the model operation parameters and the data volume before and after the improvement are greatly reduced, and the memory occupancy is greatly reduced. For comparison of model training under the data set, the training and detection effects of the model of YOLOv5m + ghost are better.
In summary, the present application has the following effects:
(1) utilize unmanned aerial vehicle to shoot the railway bridge steel construction, can acquire the better bridge steel construction remote sensing image of image quality under the condition that does not influence the line operation, it is faster than artifical collection speed, gather safelyr.
(2) The improved YOLOv5 algorithm is applied to bridge steel structure detection, so that automatic identification of defects of bridge steel structure parts is realized, and the detection efficiency is improved. Compared with the prior art, the invention is superior to YOLOv5 in the aspects of training parameters, memory occupancy and the like, and has certain application value.
(3) The invention provides a method for detecting structural defects of bridge steel, which comprises the steps of bolt loss, corrosion and corrosion of the surface of a steel structure. According to the invention, a Ghost Bottleneck module is added on the basis of the traditional YOLOv5 algorithm, and a similar feature pair in the feature extraction process is utilized, so that the inherent feature information is disclosed, and the improved YOLOv5 has great advantages in reducing the GPU occupancy rate and reducing the model parameters.
The above are merely embodiments of the present application, and the description is specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the protection scope of the present application.

Claims (10)

1.一种基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,步骤包括:1. a railway bridge steel structure defect detection method based on unmanned aerial vehicle operation, is characterized in that, step comprises: 第一步,通过搭载于无人机上的变焦相机,在距离桥梁预设距离范围的空中拍摄铁路桥梁钢结构,获得若干桥梁钢结构图像;The first step is to use the zoom camera mounted on the drone to shoot the steel structure of the railway bridge in the air within the preset distance range from the bridge, and obtain several images of the steel structure of the bridge; 第二步,对第一步所拍摄的干桥梁钢结构图像进行预处理,扩充数据集,并将扩充后数据集中的图像按照预设比例划分为训练集和测试集,利用标注工具对其中训练集中的缺陷目标进行标注;The second step is to preprocess the dry bridge steel structure images captured in the first step, expand the data set, and divide the images in the expanded data set into a training set and a test set according to a preset ratio, and use the annotation tool to train them. Centralized defect targets are marked; 第三步,将训练集中图像调整至统一尺寸,将YOLOv5模型backbone中所采用的Bottleneck结构更改为Ghost Bottleneck结构,在YOLOv5模型最后Prediction部分使用GIoU Loss作为bounding box的损失,构建改进YOLOv5模型,然后将训练数据集输入到构建所获得的改进YOLOv5模型中进行训练,比对模型输出与标注信息,调整模型参数,直至训练所获得的改进YOLOv5模型输出与标注信息相一致的检测结果;The third step is to adjust the images in the training set to a uniform size, change the Bottleneck structure used in the YOLOv5 model backbone to the Ghost Bottleneck structure, and use the GIoU Loss as the loss of the bounding box in the last Prediction part of the YOLOv5 model, build an improved YOLOv5 model, and then Input the training data set into the improved YOLOv5 model obtained by building for training, compare the model output with the label information, and adjust the model parameters until the output of the improved YOLOv5 model obtained by training is consistent with the label information. 第四步,将测试集输入到训练所确定的改进YOLOv5模型中,对铁路桥梁钢结构缺陷目标进行检测,标记并输出检测结果。In the fourth step, the test set is input into the improved YOLOv5 model determined by the training to detect the defect target of the steel structure of the railway bridge, mark and output the detection result. 2.如权利要求1所述的基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,所述预处理步骤包括对第一步所拍摄的干桥梁钢结构图像进行以下任一处理步骤或先后进行以下若干处理步骤:对图像进行旋转处理,镜像处理,调整图像对比度,调整图像亮度,调整图像增加噪声。2. The method for detecting defects of railway bridge steel structure based on drone operation as claimed in claim 1, wherein the preprocessing step comprises performing any one of the following processing on the dry bridge steel structure image captured in the first step The following steps are performed in succession: rotating the image, mirroring the image, adjusting the contrast of the image, adjusting the brightness of the image, and adjusting the image to increase noise. 3.如权利要求2所述的基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,第一步中,通过搭载于无人机上的变焦相机,在距离桥梁预设距离范围的空中拍摄铁路桥梁钢结构时,无人机距离桥梁的距离设置在50~70m之间,拍摄过程中无人机的飞行速度为2m/s,无人机拍摄图像的间隔周期为2s。3. the railway bridge steel structure defect detection method based on unmanned aerial vehicle operation as claimed in claim 2 is characterized in that, in the first step, by the zoom camera mounted on the unmanned aerial vehicle, at the distance from the bridge preset distance range. When shooting the steel structure of the railway bridge in the air, the distance between the drone and the bridge is set between 50 and 70m, the flying speed of the drone during the shooting process is 2m/s, and the interval period of the drone shooting images is 2s. 4.任意权利要求1-3所述的基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,改进YOLOv5模型中,输入端包括:Mosaic数据增强模块、自适应锚框计算模块、自适应图片缩放模块;4. the described railway bridge steel structure defect detection method based on unmanned aerial vehicle operation described in any claim 1-3, it is characterized in that, in improving YOLOv5 model, input end comprises: Mosaic data enhancement module, self-adaptive anchor frame calculation module, Adaptive image scaling module; 改进YOLOv5模型中,Ghost Bottleneck和Neck分别采用Focus结构、CSP结构和FPN+PAN结构。In the improved YOLOv5 model, Ghost Bottleneck and Neck adopt Focus structure, CSP structure and FPN+PAN structure respectively. 5.如权利要求4所述的基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,所述Ghost Bottleneck结构由两个堆叠的Ghost模块组成,其中第一个Ghost模块为增加通道数量的拓展层,第二个Ghost模块用于减少通道数量以匹配shortcut,两个Ghost模块的输入输出相连接。5. The method for detecting defects in railway bridge steel structures based on drone operation as claimed in claim 4, wherein the Ghost Bottleneck structure is composed of two stacked Ghost modules, wherein the first Ghost module is an additional channel The number of expansion layers, the second Ghost module is used to reduce the number of channels to match the shortcut, and the input and output of the two Ghost modules are connected. 6.如权利要求4所述的基于无人机作业的铁路桥梁钢结构缺陷检测方法,其特征在于,第四步对铁路桥梁钢结构缺陷目标进行检测过程中,6. The railway bridge steel structure defect detection method based on unmanned aerial vehicle operation as claimed in claim 4, is characterized in that, in the 4th step, during the detection process of railway bridge steel structure defect target, 记检测结果的交叠率为IoUresult,设定其阈值为0.5,当IoUresult>0.5认为检测到目标,否则认为为检测到目标,根据检测到的目标标记相应图像并输出。Note the overlap rate of the detection result, IoU result , and set its threshold to 0.5. When IoU result >0.5, the target is considered to be detected, otherwise, the target is considered to be detected, and the corresponding image is marked according to the detected target and output. 7.一种基于无人机作业的铁路桥梁钢结构缺陷检测系统,其特征在于,包括:7. A railway bridge steel structure defect detection system based on drone operation is characterized in that, comprising: 无人机,用于以2m/s的飞行速度,在距离铁路桥梁50-70m的空中巡航;UAV, used for cruising in the air 50-70m away from the railway bridge at a flight speed of 2m/s; 变焦相机,其搭载于无人机上,在无人机空中巡航过程中,每间隔2s拍摄铁路桥梁钢结构,获得若干桥梁钢结构图像;The zoom camera, which is mounted on the drone, takes pictures of the steel structure of the railway bridge every 2s during the aerial cruise of the drone, and obtains several images of the steel structure of the bridge; 图像存储单元,其存储有拍摄所获得的干桥梁钢结构图像,以及对拍摄所获得的桥梁钢结构图像进行预处理所获得的扩充图像;an image storage unit, which stores the image of the dry bridge steel structure obtained by shooting, and the expanded image obtained by preprocessing the image of the bridge steel structure obtained by shooting; 存储器,其存储有可在处理器上运行的计算机程序;a memory that stores a computer program executable on the processor; 处理器执行所述计算机程序时实现以下检测过程:When the processor executes the computer program, the following detection processes are implemented: 对变焦相机所拍摄的干桥梁钢结构图像进行预处理,扩充数据集,并将扩充后数据集中的图像按照预设比例划分为训练集和测试集,利用标注工具对其中训练集中的缺陷目标进行标注;然后将训练集中图像调整至统一尺寸,将YOLOv5模型backbone中所采用的Bottleneck结构更改为Ghost Bottleneck结构,在YOLOv5模型最后Prediction部分使用GIoU Loss作为bounding box的损失,构建改进YOLOv5模型,然后将训练数据集输入到构建所获得的改进YOLOv5模型中进行训练,比对模型输出与标注信息,调整模型参数,直至训练所获得的改进YOLOv5模型输出与标注信息相一致的检测结果;最后将测试集输入到训练所确定的改进YOLOv5模型中,对铁路桥梁钢结构缺陷目标进行检测,标记并输出检测结果。Preprocess the dry bridge steel structure images captured by the zoom camera, expand the data set, and divide the images in the expanded data set into a training set and a test set according to a preset ratio, and use the annotation tool to carry out the defect targets in the training set. Label; then adjust the images in the training set to a uniform size, change the Bottleneck structure used in the YOLOv5 model backbone to the Ghost Bottleneck structure, use GIoU Loss as the loss of the bounding box in the final Prediction part of the YOLOv5 model, build an improved YOLOv5 model, and then The training data set is input into the improved YOLOv5 model obtained from the construction for training, the model output and the annotation information are compared, and the model parameters are adjusted until the output of the improved YOLOv5 model obtained by training is consistent with the annotation information. Input into the improved YOLOv5 model determined by training to detect the defect target of railway bridge steel structure, mark and output the detection result. 8.如权利要求7基于无人机作业的铁路桥梁钢结构缺陷检测系统,其特征在于,处理器所获得的改进YOLOv5模型中,其输入端包括:Mosaic数据增强模块、自适应锚框计算模块、自适应图片缩放模块;8. the railway bridge steel structure defect detection system based on drone operation as claimed in claim 7, is characterized in that, in the improved YOLOv5 model that processor obtains, its input end comprises: Mosaic data enhancement module, self-adaptive anchor frame calculation module , adaptive image scaling module; 改进YOLOv5模型中,其Ghost Bottleneck和Neck分别采用Focus结构、CSP结构和FPN+PAN结构。In the improved YOLOv5 model, the Ghost Bottleneck and Neck adopt the Focus structure, the CSP structure and the FPN+PAN structure respectively. 9.如权利要求8所述的基于无人机作业的铁路桥梁钢结构缺陷检测系统,其特征在于,所述Ghost Bottleneck结构由两个堆叠的Ghost模块组成,其中第一个Ghost模块为增加通道数量的拓展层,第二个Ghost模块用于减少通道数量以匹配shortcut,两个Ghost模块的输入输出相连接。9. The steel structure defect detection system for railway bridges based on drone operation according to claim 8, wherein the Ghost Bottleneck structure is composed of two stacked Ghost modules, wherein the first Ghost module is an additional channel The number of expansion layers, the second Ghost module is used to reduce the number of channels to match the shortcut, and the input and output of the two Ghost modules are connected. 10.如权利要求8所述的基于无人机作业的铁路桥梁钢结构缺陷检测系统,其特征在于,所述改进YOLOv5模型具体选择YOLOv5m结构网络。The defect detection system for railway bridge steel structures based on drone operation according to claim 8, wherein the improved YOLOv5 model specifically selects a YOLOv5m structural network.
CN202111540584.8A 2021-12-16 2021-12-16 Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation Pending CN114266299A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111540584.8A CN114266299A (en) 2021-12-16 2021-12-16 Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111540584.8A CN114266299A (en) 2021-12-16 2021-12-16 Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation

Publications (1)

Publication Number Publication Date
CN114266299A true CN114266299A (en) 2022-04-01

Family

ID=80827416

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111540584.8A Pending CN114266299A (en) 2021-12-16 2021-12-16 Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation

Country Status (1)

Country Link
CN (1) CN114266299A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114677362A (en) * 2022-04-08 2022-06-28 四川大学 Surface defect detection method based on improved YOLOv5
CN114818828A (en) * 2022-05-18 2022-07-29 电子科技大学 Training method of radar interference perception model and radar interference signal identification method
CN114972780A (en) * 2022-04-11 2022-08-30 西北大学 Lightweight target detection network based on improved YOLOv5
CN115100452A (en) * 2022-06-16 2022-09-23 江苏数恒智能科技有限公司 An intelligent auxiliary acceptance system for distribution network engineering bench based on artificial intelligence
CN116559172A (en) * 2023-04-23 2023-08-08 兰州交通大学 A UAV-based steel bridge weld inspection method and system
CN117570911A (en) * 2024-01-15 2024-02-20 张家口市际源路桥工程有限公司 System and method for detecting construction space deviation of cast-in-situ box girder steel bars for bridge
CN118587733A (en) * 2024-08-06 2024-09-03 安徽省交通规划设计研究总院股份有限公司 A bridge structure identification and parameter extraction method for bridge PDF design drawings
CN118817599A (en) * 2024-05-11 2024-10-22 中铁桥隧技术有限公司 A method and platform for detecting defects on top slabs of dual-use highway and railway bridges

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020092635A1 (en) * 2018-10-30 2020-05-07 Frazzoli Emilio Redundancy in autonomous vehicles
CN111899227A (en) * 2020-07-06 2020-11-06 北京交通大学 Automatic railway fastener defect acquisition and identification method based on unmanned aerial vehicle operation
CN112884760A (en) * 2021-03-17 2021-06-01 东南大学 Near-water bridge multi-type disease intelligent detection method and unmanned ship equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020092635A1 (en) * 2018-10-30 2020-05-07 Frazzoli Emilio Redundancy in autonomous vehicles
CN111899227A (en) * 2020-07-06 2020-11-06 北京交通大学 Automatic railway fastener defect acquisition and identification method based on unmanned aerial vehicle operation
CN112884760A (en) * 2021-03-17 2021-06-01 东南大学 Near-water bridge multi-type disease intelligent detection method and unmanned ship equipment

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
史芳菲: "监控视频下运营车辆司机吸烟行为检测系统的设计与实现", 中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑, no. 2021, 15 September 2021 (2021-09-15), pages 1 - 58 *
武玉伟: "深度学习基础与应用", vol. 2020, 30 November 2020, 北京理工大学出版社, pages: 273 - 118 *
沈浩等: "基于深度学习的钢桁架桥螺栓病害智能识别方法", 基于深度学习的钢桁架桥螺栓病害智能识别方法, vol. 2020, no. 5, 30 September 2020 (2020-09-30), pages 608 - 615 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114677362B (en) * 2022-04-08 2023-09-12 四川大学 Surface defect detection method based on improved YOLOv5
CN114677362A (en) * 2022-04-08 2022-06-28 四川大学 Surface defect detection method based on improved YOLOv5
CN114972780A (en) * 2022-04-11 2022-08-30 西北大学 Lightweight target detection network based on improved YOLOv5
CN114818828B (en) * 2022-05-18 2024-04-05 电子科技大学 Training method of radar interference perception model and radar interference signal recognition method
CN114818828A (en) * 2022-05-18 2022-07-29 电子科技大学 Training method of radar interference perception model and radar interference signal identification method
CN115100452A (en) * 2022-06-16 2022-09-23 江苏数恒智能科技有限公司 An intelligent auxiliary acceptance system for distribution network engineering bench based on artificial intelligence
CN116559172A (en) * 2023-04-23 2023-08-08 兰州交通大学 A UAV-based steel bridge weld inspection method and system
CN117570911A (en) * 2024-01-15 2024-02-20 张家口市际源路桥工程有限公司 System and method for detecting construction space deviation of cast-in-situ box girder steel bars for bridge
CN117570911B (en) * 2024-01-15 2024-03-26 张家口市际源路桥工程有限公司 System and method for detecting construction space deviation of cast-in-situ box girder steel bars for bridge
CN118817599A (en) * 2024-05-11 2024-10-22 中铁桥隧技术有限公司 A method and platform for detecting defects on top slabs of dual-use highway and railway bridges
CN118817599B (en) * 2024-05-11 2025-06-03 中铁桥隧技术有限公司 Method and platform for detecting defects of roof of highway and railway dual-purpose bridge
CN118587733A (en) * 2024-08-06 2024-09-03 安徽省交通规划设计研究总院股份有限公司 A bridge structure identification and parameter extraction method for bridge PDF design drawings
CN118587733B (en) * 2024-08-06 2024-10-22 安徽省交通规划设计研究总院股份有限公司 A bridge structure identification and parameter extraction method for bridge PDF design drawings

Similar Documents

Publication Publication Date Title
CN114266299A (en) Method and system for detecting defects of steel structure of railway bridge based on unmanned aerial vehicle operation
CN112884760B (en) Intelligent detection method for multiple types of diseases near water bridges and unmanned ship equipment
CN111222574B (en) Target detection and classification method for ships and civilian ships based on multi-model decision-level fusion
US11410002B2 (en) Ship identity recognition method based on fusion of AIS data and video data
KR102171122B1 (en) Vessel detection method and system based on multidimensional features of scene
CN110688925B (en) Cascade target identification method and system based on deep learning
Yang et al. Deep learning‐based bolt loosening detection for wind turbine towers
CN111899227A (en) Automatic railway fastener defect acquisition and identification method based on unmanned aerial vehicle operation
CN112700444B (en) Bridge bolt detection method based on self-attention and center point regression model
CN104751486B (en) A kind of moving target relay tracking algorithm of many ptz cameras
CN111951212A (en) Method for Defect Recognition of Railway Catenary Image
CN113420819B (en) Lightweight underwater target detection method based on CenterNet
CN105930822A (en) Human face snapshot method and system
CN107194396A (en) Method for early warning is recognized based on the specific architecture against regulations in land resources video monitoring system
CN110033411A (en) The efficient joining method of highway construction scene panoramic picture based on unmanned plane
CN109145747A (en) A kind of water surface panoramic picture semantic segmentation method
CN115100554A (en) A UAV power inspection system based on intelligent vision and its detection method
CN106851229B (en) Security and protection intelligent decision method and system based on image recognition
CN110515378A (en) An intelligent target search method applied to unmanned boats
CN113160209A (en) Target marking method and target identification method for building facade damage detection
CN116309370A (en) Real-time detection method and system for apparent diseases of concrete dam
CN110427030B (en) An autonomous docking recovery method for unmanned boats based on Tiny-YOLOship target detection algorithm
CN118262145A (en) Improved YOLOv5 bridge crack detection method based on coordinate attention mechanism
CN115063838B (en) Method and system for detecting fisheye distortion image
CN117809217A (en) Method and system for scouting and beating based on real-time single-stage target recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination