CN107330931B - A method and system for detecting longitudinal displacement of rails based on image sequences - Google Patents

Abstract

The invention discloses a method for detecting the longitudinal displacement of rails based on image sequences. The method includes: S1: setting coding marks on a fixed reference structure next to the rails and on the rail waist; The image sequence is composed of on-site images of the angle and the longitudinal displacement line of the rail including all coded marks; S3: A mark detection method based on convolutional neural network, constructing a coded mark detection and positioning model, and detecting and locating the coded marks in the image sequence; S4 : Decoding the feature points of each coded mark and the sub-pixel image coordinates of the feature points; S5: Constructing the three-dimensional reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates; S6: Calculating the longitudinal displacement of the rail based on the three-dimensional reconstruction model, the present invention A system using the method is also disclosed. The invention improves the detection efficiency and accuracy of rail longitudinal displacement detection, and is simple in operation and easy to implement.

Description

A method and system for detecting longitudinal displacement of rails based on image sequences

technical field

The invention relates to the technical field of railway engineering. More specifically, it relates to a method and system for detecting longitudinal displacement of rails based on image sequences.

Background technique

The high smoothness of the track is the primary condition to ensure the safety and comfort of high-speed train operation, and the high smoothness of the track depends on the geometric state of the rail. Therefore, the detection of the geometric state of the rail is very important. Due to the thermal expansion and contraction and the longitudinal resistance of the train, the steel rail will have a certain degree of longitudinal displacement, which will easily lead to the breakage of the track or the rail, posing a huge safety hazard.

As the speed of trains continues to increase and the number of trains continues to increase, the detection of the geometric state of the rails is becoming increasingly arduous. At present, the method of manually detecting the longitudinal displacement of rails is complex, time-consuming, and low-precision, and requires the installation of fixed devices on the line in the early stage, and the burying of detection equipment beside the track, which takes up a lot of manpower and material resources, affects the operation of the line to a certain extent, and cannot To meet the requirements of modern railways, fast and accurate automatic detection has become an inevitable development trend.

Therefore, it is necessary to provide a simple, fast, safe and accurate method for detecting the longitudinal displacement of rails, so as to solve the problems of complex operation, long time consumption and low precision of manual detection of longitudinal displacement of rails at this stage.

Contents of the invention

An object of the present invention is to provide a method for detecting longitudinal displacement of rails based on image sequences. Another object of the present invention is to provide an image sequence-based rail longitudinal displacement detection system to improve the detection speed and accuracy of rail longitudinal displacement detection.

To achieve the above object, the present invention adopts the following technical solutions:

One aspect of the present invention discloses a method for detecting the longitudinal displacement of a rail based on an image sequence, the method comprising:

S1: Set coding marks on the fixed reference structure next to the rail and on the rail waist;

S2: The on-site images of the longitudinal displacement lines of the rails at different times and angles, including all coding marks, are collected by the image acquisition equipment to form an image sequence;

S3: Based on the mark detection method of the convolutional neural network, construct the code mark detection and positioning model, and detect and locate the code mark in the image sequence;

S4: Decoding the feature points of each coded mark and the sub-pixel image coordinates of the feature points;

S5: Construct the 3D reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates;

S6: Calculate the longitudinal displacement of the rail based on the three-dimensional reconstruction model.

Preferably, said fixed reference structure comprises electrified columns and/or bridge tie rods.

Preferably, the method of setting the coding mark is spraying, hanging or sticking a label.

Preferably, the image sequence includes more than three on-site images of rail longitudinal displacement lines.

Preferably, said step S3 includes:

S31: Collect a large number of on-site images of rail longitudinal displacement lines, and construct a rail longitudinal displacement image data set for training;

S32: Perform training and learning on the data set based on a convolutional neural network algorithm, and generate a coding mark detection and positioning model;

S33: Detect and locate the encoding markers in the image sequence based on the encoding marker detection and positioning model.

Preferably, the step S32 includes:

S321: Based on the on-site image data set of the rail longitudinal displacement line to be trained, obtain the labeling data of the coding mark in the image to be trained;

S322: Input the on-site image data set of the rail longitudinal displacement line and the labeling data into the convolutional neural network for training, and generate a coded mark detection and positioning model.

Preferably, step S5 includes:

S51: Estimate a camera motion parameter matrix according to the sub-pixel image coordinates of the feature points;

S52: Based on the camera motion parameter matrix corresponding to each scene image, the spatial coordinates of the feature points are estimated by triangulation;

S53: Optimizing the spatial coordinates of the feature points and the camera motion matrix through the beam adjustment method to obtain the 3D reconstruction result of the scene.

Preferably, step S6 includes:

S61: Unify the three-dimensional reconstruction models at each detection time point into the same coordinate system;

S62: Calculate the longitudinal displacement of the rail between different detection time points based on the three-dimensional reconstruction model in the same coordinate system.

Another aspect of the present invention simultaneously discloses an image sequence-based rail longitudinal displacement detection system, the system comprising:

The coded mark positioning unit is used to form an image sequence based on the fixed reference structure next to the rail and the coded mark set on the rail waist, through the image acquisition equipment to collect the on-site images of the longitudinal displacement line of the rail at different times and at different angles, including all the coded marks ;

The feature point positioning unit is used for the mark detection method based on the convolutional neural network, constructs the code mark detection and positioning model, detects and locates the code mark in the image sequence, decodes the feature points of each code mark and the sub-pixel of the feature point image coordinates;

The three-dimensional reconstruction unit is used for constructing the three-dimensional reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates of the feature points;

A displacement calculation unit, configured to calculate the longitudinal displacement of the rail based on the three-dimensional reconstruction model.

The beneficial effects of the present invention are as follows:

The technical scheme of the present invention is based on the on-site image sequence of the longitudinal displacement line of the rail from multiple perspectives collected at different detection time points, and can accurately and efficiently detect the longitudinal displacement of the rail. This technology provides fast, accurate and reliable theoretical and technical support for ensuring the safety of modern railway operations.

Description of drawings

The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings.

FIG. 1 shows a flow chart of a method for detecting longitudinal displacement of a rail based on an image sequence in the present invention.

Fig. 2 shows a schematic diagram of setting coding marks in an image sequence-based rail longitudinal displacement detection method according to the present invention.

Fig. 3 shows an example of coded marks that can be used in an image sequence-based rail longitudinal displacement detection method according to the present invention.

Fig. 4 shows a schematic diagram of on-site image acquisition of a rail longitudinal displacement line in an image sequence-based rail longitudinal displacement detection method according to the present invention.

Fig. 5 shows a schematic diagram of the three-dimensional reconstruction of the feature points of the coding marks in an image sequence-based rail longitudinal displacement detection method according to the present invention.

Fig. 6 shows an illustration of a three-dimensional reconstruction model at different detection time points in an image sequence-based rail longitudinal displacement detection method according to the present invention.

Fig. 7 shows a schematic diagram of the model transfer of the three-dimensional reconstruction models at different detection time points in the method for detecting the longitudinal displacement of rails based on image sequences in the present invention to the same coordinate system.

Detailed ways

In order to illustrate the present invention more clearly, the present invention will be further described below in conjunction with preferred embodiments and accompanying drawings. Similar parts in the figures are denoted by the same reference numerals. Those skilled in the art should understand that the content specifically described below is illustrative rather than restrictive, and should not limit the protection scope of the present invention.

As shown in Figure 1, the present invention discloses a method for detecting longitudinal displacement of rails based on image sequences, the method comprising:

S1: Code marks are set on the fixed reference structure next to the rail and on the rail waist. The reference structure and the rail waist can be fixed on electrified columns, bridge tie rods and other rails, and coding marks can be set by spraying, listing, and OEM, as shown in Figure 2. The coding marks can adopt various coding marks in photogrammetry, as shown in Figure 3.

S2: The on-site images of the longitudinal displacement line of the rail at different times and at different angles, including all coding marks, are collected by the image acquisition equipment to form an image sequence. Cameras and other image acquisition devices can be used to collect rail longitudinal displacement scene image sequences at each detection time point. Each image sequence contains at least 3 high-quality high-definition images from different perspectives, and each image contains all preset coded signs in the scene. As shown in Figure 4. The obtained on-site images of the longitudinal displacement lines of the rails can be stored in the storage device, or uploaded to the server, to be retrieved and used by subsequent devices.

S3: The image collected by the image acquisition device is greatly affected by complex factors such as weather, light, and noise, and the proportion of the detection target in the image is small and the position changes greatly. The traditional target positioning method cannot accurately locate the coded mark. . Therefore, the present invention uses a convolutional neural network-based marker detection method to construct a coded marker region location model to detect and locate the coded markers in the image sequence. Among them, the practice platform of convolutional neural network adopts Caffe, which is a deep learning framework based on C++, which supports command line, Python and MATLAB interface. The specific implementation method is as follows:

S31: Based on the obtained on-site images of the longitudinal displacement lines of the rails, a data set of longitudinal displacement images of the rails for training is constructed. The images in the data set come from a large number of on-site images of rail longitudinal displacement lines taken by inspectors.

S32: Based on the convolutional neural network method, the data set is trained and learned, and a detection and positioning model of the coded marked area is generated. Based on the on-site image data set of the longitudinal displacement line of the rail to be trained, the label data of the coding mark in the image to be trained is obtained, and then the on-site image data set of the longitudinal displacement line of the rail and the label data are input into the convolutional neural network for training to generate a code Landmark detection and localization models. In the present invention, according to the data set constructed above, based on the Caffe platform, the convolutional neural network model is used to train and learn the data set, and generate a CaffeModel file for detecting and locating coding marks. Among them, the CaffeModel file is a unique file generated based on the training of the Caffe platform. It supports command line, Python and MATLAB interfaces, and can be easily called to output detection and positioning results.

S33: Detect and locate the encoding markers in the image sequence based on the encoding marker detection and positioning model.

S4: Decoding the feature points of each coded mark and the sub-pixel image coordinates of the feature points. It can be decoded based on information such as the geometric structure and color of the coded mark, and the coded value of the feature point and the sub-pixel precise image coordinates can be obtained, as shown in Figure 5.

S5: Construct the 3D reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates, as shown in Figure 6. Step S5 may include:

S51: Estimate a camera motion parameter matrix according to the sub-pixel image coordinates of the feature points;

S52: Based on the camera motion parameter matrix corresponding to each scene image, the spatial coordinates of the feature points are estimated by triangulation;

S53: Optimizing the spatial coordinates of the feature points and the camera motion matrix through the beam adjustment method to obtain the 3D reconstruction result of the scene.

S6: Calculate the longitudinal displacement of the rail between different detection time points based on the three-dimensional reconstruction model. Using the sub-pixel image coordinates of the feature points of the coded marks on the trackside fixed reference structure in the image sequences collected at each detection time point to calculate the viewing angle conversion relationship of each image sequence. Unify the 3D reconstruction model of S5 into the same coordinate system, and use the 3D reconstruction results of the feature points on the coded marks on the rail waist in the image sequences collected at different detection time points to calculate the longitudinal displacement of the rail between different detection time points, as shown in Fig. 7. The obtained detection results of the longitudinal displacement of the rail can be stored in the storage device or uploaded to the server to be retrieved and used by subsequent devices.

The present invention also discloses an image sequence-based rail longitudinal displacement detection system, the system comprising:

The coded mark positioning unit is used to form an image sequence based on the fixed reference structure next to the rail and the coded mark set on the rail waist, through the image acquisition equipment to collect the on-site images of the longitudinal displacement line of the rail at different times and at different angles, including all the coded marks .

The feature point positioning unit is used for the mark detection method based on the convolutional neural network, constructs the code mark detection and positioning model, detects and locates the code mark in the image sequence, decodes the feature points of each code mark and the sub-pixel of the feature point image coordinates.

The three-dimensional reconstruction unit is used for estimating the camera motion parameter matrix and estimating the space coordinates of each feature point based on the sub-pixel image coordinates of the feature points, and constructing the on-site three-dimensional reconstruction model.

A displacement calculation unit, configured to calculate the longitudinal displacement of the rail between different detection time points based on the three-dimensional reconstruction model.

The present invention collects image sequences based on coded marks set on fixed reference structures and rail waists, and then uses a method based on deep learning to detect and accurately locate the coded marks to overcome image noise and irrelevant information interference. Guided by the positioning results of deep learning, the coded values and sub-pixel image coordinates of feature points are decoded and obtained based on information such as the geometric structure and color of coded marks. According to the sub-pixel image coordinates of feature points in each image sequence, 3D reconstruction is performed for each rail longitudinal displacement scene. Then use the sub-pixel image coordinates of the feature points of the coded marks on the trackside fixed reference structure in the image sequences collected at each detection time point to calculate the viewing angle conversion relationship of each image sequence, and unify the 3D reconstruction results into the same coordinate system. The three-dimensional reconstruction results of the feature points on the coded marks on the rail waist in the image sequences collected at different detection time points can finally calculate the longitudinal displacement of the rail between different detection time points.

The invention is suitable for the non-contact accurate detection of the longitudinal displacement of the rail, without the need for detection personnel to go on the track, without setting a calibration device on the line, without setting a fixed imaging device at the side of the track, without an auxiliary light source, and the cost of setting the coding mark is extremely low. No impact, the detection operation is simple and fast, and the detection accuracy is high. It can be further promoted and applied to the displacement detection of other key structural components such as switch core rails, and provides fast, accurate and reliable theoretical technical support for ensuring the safety of railway operations.

Apparently, the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those of ordinary skill in the art can also make It is impossible to exhaustively list all the implementation modes here, and any obvious changes or changes derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims (9)

1. a rail longitudinal displacement detection method based on image sequence, it is characterized in that, described method comprises: S1: Set coding marks on the fixed reference structure next to the rail and on the rail waist; S2: The on-site images of the longitudinal displacement lines of the rails at different times and angles, including all coding marks, are collected by the image acquisition equipment to form an image sequence; S3: Based on the mark detection method of the convolutional neural network, construct the code mark detection and positioning model, and detect and locate the code mark in the image sequence; S4: Decoding the feature points of each coded mark and the sub-pixel image coordinates of the feature points; S5: Construct the 3D reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates; S6: Calculate the longitudinal displacement of the rail based on the three-dimensional reconstruction model. 2. The method of claim 1, wherein the fixed reference structures include electrified columns and/or bridge tie rods. 3. The method according to claim 1, characterized in that, the way of setting the coding mark is spraying, hanging or sticking a label. 4. The method according to claim 1, wherein the image sequence includes more than three on-site images of rail longitudinal displacement lines. 5. The method according to claim 1, wherein said step S3 comprises: S31: Collect a large number of on-site images of rail longitudinal displacement lines, and construct a rail longitudinal displacement image data set for training; S32: Perform training and learning on the data set based on a convolutional neural network algorithm, and generate a coding mark detection and positioning model; S33: Detect and locate the encoding markers in the image sequence based on the encoding marker detection and positioning model. 6. The method according to claim 5, wherein the step S32 comprises: S321: Based on the on-site image data set of the rail longitudinal displacement line to be trained, obtain the labeling data of the coding mark in the image to be trained; S322: Input the on-site image data set of the rail longitudinal displacement line and the labeling data into the convolutional neural network for training, and generate a coded mark detection and positioning model. 7. The method according to claim 1, wherein step S5 comprises: S51: Estimate a camera motion parameter matrix according to the sub-pixel image coordinates of the feature points; S52: Based on the camera motion parameter matrix corresponding to each scene image, the spatial coordinates of the feature points are estimated by triangulation; S53: Optimizing the spatial coordinates of the feature points and the camera motion matrix through the beam adjustment method to obtain the 3D reconstruction result of the scene. 8. The method according to claim 1, wherein step S6 comprises: S61: Unify the three-dimensional reconstruction models at each detection time point into the same coordinate system; S62: Calculate the longitudinal displacement of the rail between different detection time points based on the three-dimensional reconstruction model in the same coordinate system. 9. A rail longitudinal displacement detection system based on image sequences, characterized in that the system comprises: The coded mark positioning unit is used to form an image sequence based on the fixed reference structure next to the rail and the coded mark set on the rail waist, through the image acquisition equipment to collect the on-site images of the longitudinal displacement line of the rail at different times and from different angles, including all the coded marks ; The feature point positioning unit is used for the mark detection method based on the convolutional neural network, constructs the code mark detection and positioning model, detects and locates the code mark in the image sequence, decodes the feature points of each code mark and the sub-pixel of the feature point image coordinates; The three-dimensional reconstruction unit is used for constructing the three-dimensional reconstruction model of the rail longitudinal displacement line site based on the sub-pixel image coordinates of the feature points; A displacement calculation unit, configured to calculate the longitudinal displacement of the rail based on the three-dimensional reconstruction model.