CN108389205B - A method and device for monitoring foreign objects on rails based on images of space-based platforms - Google Patents

Abstract

The invention provides a rail foreign matter monitoring method and device based on an empty foundation platform image, wherein the method comprises the following steps: acquiring a picture to be processed, wherein the picture to be processed is a rail picture shot by a low-altitude unmanned machine; obtaining effective gradient information of a picture to be processed; coding the effective gradient information according to the type of the effective gradient information to obtain a first characteristic; obtaining a second characteristic according to the hue, saturation and transparency HSV color model of the picture to be processed; and judging whether foreign matters exist in the rail in the picture to be processed or not according to the first characteristic and the second characteristic. According to the rail foreign matter monitoring method and device based on the air-based platform image, whether foreign matters exist on the rail is judged by combining the effective gradient information and the color information of the image, and the monitoring efficiency of the rail foreign matters is improved.

Description

A method and device for monitoring foreign objects on rails based on images of space-based platforms

technical field

The invention relates to aviation monitoring technology, in particular to a method and device for monitoring foreign objects on rails based on an air-based platform image.

Background technique

Rail transit is the main way for the country's passenger and freight transportation, and has very important strategic significance. Among them, railway transportation plays a particularly important role. Due to the large population in my country, the high-traffic railway traffic requires more rigorous security inspections to ensure the safety of passengers.

In the prior art, with the wide application of low-altitude UAVs in the field of inspection and surveillance, more and more air-based platforms using low-altitude UAVs are used for inspections along railway lines to save manpower and material costs. The rail maintenance personnel monitor the condition of the railway through the rail pictures returned by the low-altitude drone of the air-based platform, and conduct on-site maintenance in time when foreign objects are found on the rail.

Using the existing technology, in order to ensure the safe operation of the railway, the low-altitude drone inspection of the air-based platform needs to feed back the information along the railway more accurately and in a timely manner. Therefore, the monitoring efficiency of foreign objects on the rails is low and requires a lot of manpower and material resources.

SUMMARY OF THE INVENTION

The invention provides a method and device for monitoring foreign objects on a rail track based on an image of an air-based platform, which improves the monitoring efficiency of foreign objects on the rail track.

The present invention provides a method for monitoring foreign objects on a railway track based on an image of an air-based platform, comprising:

obtaining a picture to be processed, where the picture to be processed is a picture of a railroad track photographed by a low-altitude drone;

obtaining the effective gradient information of the picture to be processed;

The first feature is obtained by encoding the effective gradient information according to the type of the effective gradient information;

Obtain the second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed;

According to the third feature obtained after the fusion of the first feature and the second feature, it is determined whether there is a foreign object on the rail in the image to be processed.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on a railway track based on an image of an air-based platform, the obtaining effective gradient information of the image to be processed includes:

establishing an integral image of the picture to be processed;

Passing the integral image through a box filter to establish a scale space;

locating feature points in the scale space;

By constructing feature point descriptors, the effective gradient information of the picture to be processed is obtained.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on rails based on an image of an air-based platform, the first feature is obtained by encoding the effective gradient information according to the type of the effective gradient information, including:

The effective gradient information is clustered by a clustering algorithm, and the cluster center is used as a basic codeword;

According to the basic codeword, a bag-of-words model is used to encode the effective gradient information to obtain the first feature in a fixed encoding format.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on rails based on an air-based platform image, the second feature is obtained according to the hue, saturation, and transparency HSV color model features of the picture to be processed, including: :

Obtain the HSV color model data of the picture to be processed;

Statistics of the HSV color model data are performed according to the color histogram to obtain the HSV color model feature, which is used as the second feature.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on a railway track based on an image of an air-based platform, it is determined whether there is a railway track in the image to be processed according to the first feature and the second feature Foreign objects, including:

Performing feature fusion on the first feature and the second feature to obtain a third feature;

The classifier judges whether the rails in the picture to be processed have foreign objects through the third feature, and the classifier includes: the third feature of the rail pictures with foreign objects and the first feature of the rail pictures without foreign objects. Three characteristics.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on a railway track based on an image of an air-based platform, before acquiring the image to be processed, the method further includes:

Obtaining the third features of N pictures of the rails with foreign objects and the third features of M pictures of the tracks without foreign objects, wherein N and M are positive integers;

The third features of the N pictures of the railroad tracks with foreign objects and the third features of the M pictures of the railroad tracks without foreign objects are stored in the classifier.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on a railway track based on an image of a space-based platform, the classifier is a support vector machine (SVM).

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on a railway track based on an image of an air-based platform, it is determined whether there is a railway track in the image to be processed according to the first feature and the second feature After the foreign body, it also includes:

If it is determined that there is a foreign object on the rail in the picture to be processed, the attribute of the foreign object is acquired through the third feature.

In an embodiment of the present invention, in the above-mentioned method for monitoring foreign objects on rails based on an image of an air-based platform, the properties of the foreign objects include one or more of the following: type, size, shape and color of the foreign object.

The present invention provides a device for monitoring foreign objects on a railway track based on an image of an air-based platform, comprising: an acquisition module, wherein the acquisition module is used to acquire a picture to be processed, and the picture to be processed is a picture of a rail track photographed by a low-altitude drone;

a feature extraction module, the processing module is used to obtain the effective gradient information of the to-be-processed picture;

The feature extraction module is further configured to perform encoding processing on the effective gradient information according to the type of the effective gradient information to obtain a first feature;

The feature extraction module is further configured to obtain a second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed;

A classification module, the classification module is configured to determine whether there is a foreign object in the rail in the to-be-processed picture according to the third feature obtained after the fusion of the first feature and the second feature.

The present invention provides a method and device for monitoring foreign objects on a railway track based on an image of an air-based platform, wherein the method includes: obtaining a picture to be processed, and the picture to be processed is a picture of a railway track photographed by a low-altitude drone; obtaining effective gradient information of the picture to be processed; Types of Effective Gradient Information Encoding and processing the effective gradient information to obtain the first feature; obtaining the second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed; judging the rails in the picture to be processed by the first feature and the second feature Is there any foreign body. The invention provides a method and device for monitoring foreign objects on a rail based on an image of an air-based platform, which combines the effective gradient information and color information of the picture to determine whether there is a foreign object on the rail, thereby improving the monitoring efficiency of the foreign object on the rail.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic flowchart of Embodiment 1 of a method for monitoring foreign objects on a railway track based on an image of an air-based platform of the present invention;

FIG. 2 is a schematic structural diagram of Embodiment 1 of an apparatus for monitoring foreign objects on a railway track based on an image of an air-based platform according to the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can, for example, be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

The technical solutions of the present invention will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

FIG. 1 is a schematic flowchart of Embodiment 1 of a method for monitoring foreign objects on a railway track based on an image of an air-based platform of the present invention. As shown in FIG. 1 , a method for detecting foreign objects based on an image of a space-based platform provided in this embodiment includes:

S101: Acquire a to-be-processed picture, where the to-be-processed picture is a rail picture photographed by a low-altitude drone.

Specifically, the execution body of this embodiment may be a low-altitude UAV of an air-based platform, and the air-based platform is used to monitor foreign objects on the rails through at least one UAV; the execution body of this embodiment may also be able to use the Internet and In other ways, any one or a combination of any one or more of terminal equipment (Terminal), user equipment (User Equipment), and server equipment, etc., for obtaining the rail pictures taken by the space-based platform. The terminal device may be a desktop computer (computer), a notebook computer (notebook), a tablet computer (PAD), and the like. The user equipment may be a smart phone, a smart watch, smart glasses, and the like. It should be understood that the above examples are only for illustration and should not constitute a specific limitation.

In this step, when the low-altitude UAV conducts inspections on the railway track section to be monitored, the monitored railway track is continuously photographed to obtain continuous railway track pictures. After the low-altitude drone captures the rail image, the low-altitude drone can directly use it as the image to be processed; alternatively, the low-altitude drone uploads the rail image to the server for storage, and then obtains it from image processing equipment such as terminals and user equipment. The rail image stored in the server is used as the image to be processed.

S102: Obtain effective gradient information of the picture to be processed.

Specifically, the effective gradient information of the picture to be processed obtained in S101 is obtained, wherein optionally, the effective gradient information in the picture to be processed may be extracted by a gradient feature extractor.

Optionally, S102 extracting the effective gradient of the picture to be processed may include the following steps:

S1021: Create an integral image of the picture to be processed.

Specifically, the integral image established in this step is an image obtained by integral calculation of the image to be processed, and each point of the integral image is expressed as the pixel sum of the rectangular area from the origin to the point of the original image. The reason why the establishment of the integral image can be accelerated The calculation speed is because after we traverse the integral image of the entire image, the sum of the pixels of any rectangular area in the original image can be completed by addition and subtraction, regardless of the area of the rectangle. The larger the rectangle, the more savings more computation time.

S1022: Pass the integral image through a box filter to establish a scale space.

Specifically, in this step, the scale space is established by the box filter, and the box filter is used to approximate the Gaussian kernel function, so that the convolution templates are all composed of simple rectangles. The introduction of integral image solves the problem of fast calculation in rectangular area, and the approximation of box filter greatly improves the calculation speed. In order to ensure the scale invariance of image matching, it is necessary to layer the image, establish the scale space of the image, and then search for feature points on images of different scales. The establishment of the scale space in this embodiment is to keep the size of the image to be processed unchanged, and filter the integral image calculated from the image to be processed by changing the size of the box filter, thereby forming the scale space of the image.

S1023: Locate feature points in the scale space.

Specifically, through the scale space established in S102, a fast Hessian matrix is used on each image layer of the scale space to detect the extreme point of the image. For any point (x, y) in the space, the scale in the corresponding scale space is σ, then the definition of the Hessian matrix is as follows:

卷积的结果，其中g为高斯函数。where L _xx (xσ), L _xy (xσ), and L _yy (xσ) are the points on the image and the Gaussian second-order partial derivatives respectively

The result of the convolution, where g is a Gaussian function.

At the same time, in order to obtain the stable position and scale value of the feature point, the scale space can be interpolated, so that the position value of the feature point and the scale value of the feature point are obtained.

S1024: Obtain effective gradient information of the image to be processed by constructing feature point descriptors.

Specifically, in this step, the main direction of the feature point is obtained first, which can ensure the rotation invariance of the algorithm, and then the neighborhood of the feature point is rotated to the main direction to describe the feature point. In order to make the matching of images have rotation invariance, the concept of principal direction is introduced. The calculation of the main direction is to take the feature point as the center, take a circular area with a radius of 6s (s is the scale value of the feature point) around the feature point, and calculate the Haar wavelet response of the pixels in the neighborhood in the x and y directions. value. A certain weight coefficient is assigned to the calculated response value according to the distance, and then histogram statistics are performed on the weighted response value. Statistics start from the x-axis, and a new vector is calculated by adding the Haar wavelet response values within 60 degrees of the circular area. Calculate the vector in the same way every 5 degrees, traverse the entire circular area, you can get 72 new vectors. We choose the longest vector direction as the main direction of this feature point.

For the detected feature points, take the feature point as the center, select a 20S*20S area within the neighborhood of the center point, and then rotate the main direction of the area to the main direction of the feature point. In order to make better use of the spatial information of the image, the 20S*20S area is divided into 4*4 total 16 sub-areas, so that the pixel value of each sub-area is 5S*5S. Finally, the feature points are described by counting the Haar wavelet response values of the pixels, and the effective gradient information of the image to be processed is obtained. Exemplarily, assuming that a picture can obtain P feature points through detection, and each feature point can be described in the above way to obtain a 128-dimensional feature descriptor, then we can use a P*128-dimensional one-dimensional vector to Represents the effective gradient information of the image to be processed.

The method for calculating the effective gradient information provided in this example is only an example, and it can be known by referring to a known method for calculating the effective gradient information in the art for places not listed. It should be noted that, in this step, the effective gradient information of the to-be-processed picture may also be calculated by other calculation methods commonly used by those skilled in the art, which is not specifically limited in this embodiment.

S103: Encoding the effective gradient information according to the type of the effective gradient information to obtain the first feature.

Specifically, in this step, since the dimensions and lengths of the effective gradient information calculated from different pictures are different, if the effective gradient information needs to be used as a feature of image processing and classification, it is necessary to calculate the different dimensions of the different pictures. , the effective gradient information of the length is processed, so that the effective gradient information of all pictures has the same dimension and length, and can be compared and classified together. Optionally, the method of processing the effective gradient information may be to first classify the type of the effective gradient information, and then perform encoding processing corresponding to the type to the effective gradient information according to the classification situation, and obtain the encoded effective gradient information as the first. feature. Among them, classification is to determine how effective gradient information is encoded.

Optionally, a possible implementation manner of this step S103 is:

S1031: Cluster the effective gradient information through a clustering algorithm, and use the cluster center as a basic codeword;

S1032: According to the basic codeword, a bag-of-words model is used to encode the effective gradient information to obtain a first feature of a fixed encoding format.

Specifically, according to the K-means clustering method, the total number of clustering categories can be preset as 1000, and the first feature is clustered by measuring the distance between the feature and the feature to obtain the cluster center as the basic codeword. A bag-of-words model is then used to encode the effective gradient information from the base codewords. The effective gradient information obtained from each image has different dimensions. Through the bag-of-words model clustering, each feature point is assigned to the corresponding cluster center, and then the histogram is calculated based on the cluster center to obtain a 1000-dimensional feature vector. That is the first feature we want to extract. Next, let's introduce a specific processing example.

个特征点，其中P是每张图片中特征点的数目。使用K-means聚类方法对上述特征点进行聚类操作，指定类别总数为1000，便可以得到1000个类别，每个类别包含若干个特征点。通过计算每个类别中所有的128维特征描述子的均值，得到每个类别的聚类中心，可以用一个128维的一维向量表示。至此，我们可以得到1000个聚类中心。随后，这些聚类中心作为基本字典，采用词袋模型的思想，对每张图片进行编码。具体的，假设图片中含有P个特征点，统计这P个特征点属于上述1000个类别各自的频率，最后可以得到一个1000维的一维向量作为最终编码后的码字共后续处理使用。Assuming that the sample contains 1w pictures taken by low-altitude drones, there are 7,000 pictures with foreign objects and 3,000 pictures without foreign objects. The gradient information in each image can be extracted by the gradient feature extractor, which is represented by several 128-dimensional feature descriptors. Thus, we can get

feature points, where P is the number of feature points in each image. Use the K-means clustering method to cluster the above feature points, and specify the total number of categories to be 1000, then 1000 categories can be obtained, and each category contains several feature points. By calculating the mean of all 128-dimensional feature descriptors in each category, the cluster center of each category can be obtained, which can be represented by a 128-dimensional one-dimensional vector. So far, we can get 1000 cluster centers. These cluster centers are then used as a basic dictionary to encode each image using the idea of the bag-of-words model. Specifically, it is assumed that the picture contains P feature points, and the frequencies of the P feature points belonging to the above 1000 categories are counted. Finally, a 1000-dimensional one-dimensional vector can be obtained as the final encoded codeword for subsequent processing.

The clustering algorithm and the bag-of-words model method provided in this example are only examples, and the places not listed can be known by referring to known methods in the art. It should be noted that, in this step, the effective gradient information can also be calculated in a unified dimension through other calculation methods commonly used by those skilled in the art, which is not specifically limited in this embodiment.

S104: Obtain the second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed.

Specifically, in this step, the color features of the picture to be processed are processed, wherein, optionally, this step specifically includes:

S1041: Acquire HSV color model data of the image to be processed;

S1042: Perform statistics on the HSV color model data according to the color histogram to obtain the HSV color model feature as the second feature.

Among them, the HSV color model is a color model for visual perception, including three components H, S, V corresponding to the hue, saturation and brightness of the color signal, and the HSV color model can be represented by an inverted cone. The magnitude from the long axis represents saturation, the long axis represents brightness, and the angle around the long axis represents color. Since the perceived color difference is proportional to the Euclidean distance. Therefore HSV is more suitable for human perception. Color histogram is a common method to represent color features, but the amount of data in the histogram is too large, so a quantized histogram is generally needed to simplify color features. To this end, the color threshold must be quantized first, and the quantization method is as follows:

Construct one-dimensional eigenvectors according to G=HQ _S Q _V +SQ _V +V, where Q _S Q _V are respectively equal to 3, so G=9H+3S+V. In this way, the color feature is quantified into an integer of 0-71, which is convenient for the histogram to be counted as a one-dimensional feature vector, which is recorded as the second feature.

The calculation method of the HSV color model provided in this example is only an example, and the parts not listed can be known by referring to the known method for calculating the HSV color model in the art. It should be noted that, in this step, the HSV color model of the to-be-processed picture may also be calculated by other calculation methods commonly used by those skilled in the art, which is not specifically limited in this embodiment.

S105: According to the first feature and the second feature, determine whether there is a foreign object on the rail in the image to be processed.

Specifically, in a possible implementation manner of this embodiment, the third feature is obtained by feature fusion of the first feature of the to-be-processed picture obtained in S103 and the second feature of the to-be-processed picture obtained in S104. Optionally, since the dimension of the first feature and the dimension of the second feature of different pictures are the same, the array of the first feature and the array of the second feature can be obtained by combining the array of the first feature and the second feature. An array of the third feature of the array and the second feature array. Alternatively, it is also possible to fuse the first feature and the second feature by adding, subtracting, or multiplying, as long as a third feature that can characterize the first feature and the second feature at the same time can be obtained, and the third feature of all pictures can be obtained. The dimensions and forms are the same, and are not limited here.

Specifically, in this step, a classifier can be used to determine whether there is a foreign object in the rail in the picture to be processed according to the third feature, wherein the classifier stores: third feature.

For example, the set A of the classifier stores the third features of N normal rail pictures without foreign objects, and the N third features in the set A are N normal rail pictures taken by low-altitude drones. According to the above implementation The N third features calculated in S101 to S105 in the example, where N is a positive integer. It is possible to select N pictures of the rails taken by low-altitude drones without foreign objects on the rails, and put the third features obtained from these pictures into the set A. Since the N third features in the set A do not contain foreign objects, are all within the same range. At the same time, the set B of the classifier stores M third features of the pictures of the rails with foreign objects, for example, the set B of the classifier stores the third features of the pictures of M railway tracks with foreign objects, the M in the set B The third feature is the M third features calculated from S101 to S105 in the above-mentioned embodiment, where M is a positive integer, and M and N may be the same or different. M low-altitude UAV images of the rails with foreign objects can be selected, and the third features obtained from these pictures are put into the set B. Since there are foreign objects in the M third features in the set B, they are in the set B. The range is different from that of the third feature in set A, and if the type of foreign object is the same, the third feature in set B is within the same range.

In S105, after obtaining the third feature of the picture to be processed through the above steps, the classifier compares the third feature of the picture to be processed with the third feature in the set A and set B according to the machine learning algorithm, and judges the third feature of the set to be processed. The third feature of the picture belongs to set A or set B. If the third feature of the picture to be processed is more similar to the third feature in set A in the comparison, it is judged that there is no foreign matter on the rails in the picture to be processed; accordingly, If the third feature of the to-be-processed picture is more similar to the third feature in the set B in the comparison, it is determined that there is a foreign object on the rail in the to-be-processed picture.

Optionally, in the foregoing embodiment, the classifier used in step S105 is a support vector machine (Support Vector Machine, SVM for short) linear classifier.

In addition, an embodiment of the present invention also provides an apparatus for dynamic scene classification with multi-feature fusion, which is used to execute the above method embodiments, and has the same technical features and technical effects, which will not be repeated.

Optionally, in the above embodiment, after S105, the method further includes: if it is determined that there is a foreign object on the rail in the picture to be processed, acquiring the attribute of the foreign object through the third feature.

Specifically, when it is determined in S105 that there is a foreign object in the rail in the to-be-processed picture, the type of the foreign object can be further identified by the classifier. The attribute of the foreign object may be parameter information such as the type, size, shape, color, and moving speed of the foreign object. For example: there are pictures of livestock (such as cattle and sheep) entering in the rail pictures collected by low-altitude drones, these pictures are calculated through the above steps to obtain the third feature and stored in the set C of the classifier; similarly, there are traffic The pictures entered by tools (such as cars and motorcycles) are stored in the set D of the classifier after calculating the third feature of these pictures. Then when the classifier receives the third feature of the picture to be processed, it classifies the third feature of the picture to be processed according to the third feature in the set C and the set D, and it is concluded that the type of foreign objects on the rails in the picture to be processed is livestock or traffic. tool. Optionally, the judgment on the properties of foreign objects in this embodiment can be implemented by using two different classifiers or the same classifier from the judgment on whether there are foreign objects in the foregoing embodiment. If implemented by the same classifier, the above set A All to set D are stored in this classifier. The above distance is an example of the types of foreign object properties, and other parameters are implemented in the same manner, and will not be repeated here. It should be noted that the moving speed of the foreign object can be obtained by combining pictures taken at different times.

To sum up, the present embodiment provides a method for detecting foreign objects on rails based on an image of an air-based platform. By acquiring a picture to be processed, the picture to be processed is a rail picture taken by a low-altitude drone; obtaining the effective gradient information of the picture to be processed; Types of Effective Gradient Information Encoding and processing the effective gradient information to obtain the first feature; obtaining the second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed; judging the rails in the picture to be processed according to the first feature and the second feature Is there any foreign body. Therefore, by combining the effective gradient information and color information of the picture, it is judged whether there is a foreign body on the rail, and the monitoring efficiency of the foreign body on the rail is improved. At the same time, in an embodiment of a method for monitoring foreign objects on a railway track based on an air-based platform image provided in an embodiment of the present invention, a gradient feature extractor is used to extract the gradient features of the pictures to be classified, and then the K-means clustering method and the word The bag model obtains high-level gradient features according to the gradient features; at the same time, the color feature extractor is used to extract the color features of the images to be classified, and the two features are fused to monitor and classify the foreign objects on the rails. When the picture to be classified is identified as a picture containing foreign objects, an early warning is issued. Not only the gradient information in the picture, but also the color information in the picture is considered, making the classification more accurate.

FIG. 2 is a schematic structural diagram of Embodiment 1 of an apparatus for monitoring foreign objects on a railway track based on an image of an air-based platform according to the present invention. As shown in FIG. 2 , the apparatus for monitoring foreign objects on rails based on drone images provided in this embodiment includes: an acquisition module 201 , a feature extraction module 202 and a classification module 203 . Among them, the acquisition module 201 is used to acquire the picture to be processed, and the to-be-processed picture is a rail picture taken by a low-altitude drone; the feature extraction module 202 is used to acquire the effective gradient information of the to-be-processed picture; the feature extraction module 202 is also used to obtain the effective gradient information according to the effective gradient The type of information encodes the effective gradient information to obtain the first feature; the feature extraction module 202 is also used to obtain the second feature according to the hue, saturation, and transparency HSV color model of the picture to be processed; the classification module 203 is used to obtain the second feature according to the The first feature and the second feature are used to determine whether there are foreign objects on the rails in the to-be-processed picture.

The apparatus provided in this embodiment is used to execute the method provided in the embodiment shown in FIG. 1 , and its implementation manner is the same as the principle, and details are not described again.

Optionally, in the above-mentioned embodiment, the feature extraction module is specifically used to establish an integral image of the picture to be processed; pass the integral image through a box filter to establish a scale space; locate the feature points of the scale space; Get the effective gradient information of the image to be processed.

Optionally, in the above embodiment, the feature extraction module is specifically configured to cluster the effective gradient information through a clustering algorithm, and use the cluster center as a basic codeword; use a bag-of-words model to perform the effective gradient information according to the basic codeword. The encoding process is performed to obtain the first feature of the fixed encoding format.

Optionally, in the above-mentioned embodiment, the feature extraction module is specifically configured to obtain HSV color model data of the picture to be processed; the HSV color model data is counted according to the color histogram to obtain HSV color model features as the second feature.

Optionally, in the above-mentioned embodiment, the classification module is specifically configured to perform feature fusion on the first feature and the second feature to obtain a third feature; the classifier judges whether there is a foreign object in the rails in the picture to be processed by using the third feature, and classifies the features. The device includes: a third feature of a picture of a railroad track with foreign objects and a third feature of a picture of a railroad track without foreign objects.

Optionally, in the above-mentioned embodiment, the acquisition module is further configured to acquire the third features of N rail pictures with foreign objects and the third features of M rail pictures without foreign objects, wherein N and M are positive integers; The third features of the N pieces of rail pictures with foreign objects and the third features of the M pieces of rail pictures without foreign objects are stored in the classifier.

Optionally, in the above embodiment, the classifier is a support vector machine (SVM).

Optionally, in the above embodiment, the classification module is further configured to obtain the attribute of the foreign object through the third feature if it is determined that there is a foreign object in the rail in the picture to be processed.

Optionally, in the above-mentioned embodiment, the classification module is further configured to obtain the location information of the low-altitude drone when the image to be processed is photographed by the low-altitude drone if it is determined that there is a foreign object on the rail in the to-be-processed picture, and send the carrying position to the alarm server. Informational warning signal.

The apparatus provided in this embodiment is used to execute the method provided in the foregoing embodiment, and the implementation manner and principle thereof are the same, and are not repeated here.

An embodiment of the present invention further provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for monitoring foreign objects on a rail track based on an image of an air-based platform in any of the above embodiments is executed. The storage medium in this embodiment is a computer-readable storage medium.

An embodiment of the present invention further provides an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any of the foregoing embodiments by executing the executable instructions A method for monitoring foreign objects on rails based on images of space-based platforms.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. scope.

Claims (7)

1. A rail foreign matter monitoring method based on an empty foundation platform image is characterized by comprising the following steps:

acquiring a picture to be processed, wherein the picture to be processed is a rail picture shot by a low-altitude unmanned machine;

obtaining effective gradient information of the picture to be processed;

coding the effective gradient information according to the type of the effective gradient information to obtain a first characteristic;

obtaining a second characteristic according to the hue, saturation and transparency HSV color model of the picture to be processed;

judging whether foreign matters exist in the rail in the picture to be processed or not according to the first characteristic and the second characteristic;

the obtaining effective gradient information of the picture to be processed comprises:

establishing an integral image of the picture to be processed;

establishing a scale space by the integral image through a box filter;

locating feature points of the scale space;

obtaining effective gradient information of the picture to be processed by constructing a feature point descriptor;

the judging whether foreign matters exist in the rail in the picture to be processed according to the first characteristic and the second characteristic comprises the following steps:

performing feature fusion on the first feature and the second feature to obtain a third feature, wherein the feature fusion is performed in a mode of combining, adding, subtracting or multiplying the first feature and the second feature;

judging whether foreign matters exist in the rail in the picture to be processed through the third features by a classifier, wherein the classifier comprises: the third feature of a rail picture with a foreign object present and the third feature of a rail picture without a foreign object present;

the encoding the effective gradient information according to the type of the effective gradient information to obtain a first characteristic includes:

clustering the effective gradient information through a clustering algorithm, and taking a clustering center as a basic code word;

and coding the effective gradient information by adopting a bag-of-words model according to the basic code words to obtain the first characteristic of a fixed coding format.

2. The method as claimed in claim 1, wherein the obtaining a second feature according to the hue, saturation and transparency HSV color model feature of the to-be-processed picture comprises:

acquiring HSV color model data of the picture to be processed;

and counting the HSV color model data according to a color histogram to obtain the HSV color model characteristic as the second characteristic.

3. The method according to claim 1, wherein before the obtaining the picture to be processed, further comprising:

acquiring the third characteristics of N rail pictures with foreign matters and the third characteristics of M rail pictures without foreign matters, wherein N and M are positive integers;

and storing the third characteristics of the N rail pictures with the foreign matters and the third characteristics of the M rail pictures without the foreign matters into the classifier.

4. The method of claim 3, wherein the classifier is a Support Vector Machine (SVM).

5. The method according to any one of claims 1 to 4, wherein after determining whether foreign objects exist on the rail in the to-be-processed picture according to the first feature and the second feature, the method further comprises:

and if the rail in the picture to be processed is judged to have the foreign matter, acquiring the attribute of the foreign matter through the third characteristic.

6. The method of claim 5, wherein the attributes of the foreign object include one or more of: the type, size, shape and color of the foreign matter.

7. A rail foreign matter monitoring device based on air-based platform images, comprising:

the acquisition module is used for acquiring a picture to be processed, and the picture to be processed is a rail picture shot by a low-altitude unmanned machine;

the processing module is used for acquiring effective gradient information of the picture to be processed;

the characteristic extraction module is used for coding the effective gradient information according to the type of the effective gradient information to obtain a first characteristic;

the feature extraction module is further used for obtaining a second feature according to the hue, saturation and transparency HSV color model of the picture to be processed;

the classification module is used for judging whether foreign matters exist in the rail in the picture to be processed according to the first characteristic and the second characteristic;

the processing module is specifically used for establishing an integral image of the picture to be processed; establishing a scale space for the integral image through a box filter; locating feature points of the scale space; obtaining effective gradient information of the picture to be processed by constructing a feature point descriptor;

the classification module is specifically configured to perform feature fusion on the first feature and the second feature to obtain a third feature, where the feature fusion is performed in a manner of combining, adding, subtracting, or multiplying the first feature and the second feature;

judging whether foreign matters exist in the rail in the picture to be processed through the third features by a classifier, wherein the classifier comprises: the third feature of a rail picture with a foreign object present and the third feature of a rail picture without a foreign object present;

the characteristic extraction module is specifically used for clustering the effective gradient information through a clustering algorithm, and taking a clustering center as a basic code word;

and coding the effective gradient information by adopting a bag-of-words model according to the basic code words to obtain the first characteristic of a fixed coding format.

Images