CN117011828A - Train obstacle detecting system - Google Patents
Train obstacle detecting system Download PDFInfo
- Publication number
- CN117011828A CN117011828A CN202310903980.5A CN202310903980A CN117011828A CN 117011828 A CN117011828 A CN 117011828A CN 202310903980 A CN202310903980 A CN 202310903980A CN 117011828 A CN117011828 A CN 117011828A
- Authority
- CN
- China
- Prior art keywords
- foreign matter
- value
- foreign
- depth value
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
- G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/225—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on a marking or identifier characterising the area
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The invention relates to the technical field of image processing, and discloses a train obstacle detection system, which is technically characterized by comprising a camera module, a foreign matter intrusion judging module, an image processing module, a laser radar and a millimeter wave radar; the camera module is configured with a visual sensor and a region segmentation strategy, the visual sensor generates image data in real time, the image data is subjected to region segmentation through the region segmentation strategy, and segmented image information containing an intrusion region and a warning region is generated; the foreign object intrusion judgment module is configured with a first foreign object calibration strategy and a first threshold; the image processing module comprises a first foreign matter identification model, the image processing module acquires first foreign matter information, and the calibration part is identified, classified and checked through the first foreign matter identification model.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a train obstacle detection system.
Background
Foreign matter intrusion refers to the intrusion of objects outside of the railroad facility into the railroad line, such as pedestrians, automobiles, falling rocks, and fallen trees. This poses a threat to the safety of the railway line and train operation. The method is mainly applied to road-to-railway sections, road-to-railway parallel adjacent sections, construction sections nearby, railway main lines and mountain areas where collapse and collapse of stone sections possibly occur, so that the method has the characteristics of burstiness, irregularity and unpredictability;
the image detection is gradually used as an obstacle detection means of a train due to the characteristics of simple installation, small maintenance workload and relatively perfect detection function, in the obstacle recognition process in the prior art, an image is adopted by a visual sensor, then the obstacles in the image are recognized in a roll neural network and other modes, the calculated amount of the obstacle recognition mode is relatively large, and if all the obstacles are measured and measured in the mode, the requirements on hardware are relatively high and the recognition time is relatively long, so that the problem of relatively low timeliness is caused.
Disclosure of Invention
In view of the shortcomings of the prior art, an object of the present invention is to provide a train obstacle detection system for overcoming the above-mentioned drawbacks of the prior art.
In order to achieve the above purpose, the present invention provides the following technical solutions: a train obstacle detection system comprises a camera module, a foreign matter intrusion judging module, an image processing module, a laser radar and a millimeter wave radar;
the camera module is configured with a visual sensor and a region segmentation strategy, the visual sensor generates image data in real time, the image data is subjected to region segmentation through the region segmentation strategy, and segmented image information containing an intrusion region and a warning region is generated;
the foreign object intrusion judgment module is configured with a first foreign object calibration strategy and a first threshold, and the first foreign object calibration strategy comprises:
s1: acquiring the segmentation image information, and calling the laser radar to sequentially scan the intrusion region along the extending direction of the intrusion region to generate a continuous actual depth value and a continuous predicted depth value in the intrusion region;
s2: comparing the continuous actual depth value with the continuous predicted depth value, if the comparison result is larger than a first threshold value, performing semantic segmentation, extracting a foreign object region, and calibrating to generate first foreign object information, wherein the first foreign object information reflects the actual position of the invasive region in the calibration;
the image processing module comprises a first foreign matter identification model, the image processing module acquires the first foreign matter information, and the calibration part is identified and classified and checked through the first foreign matter identification model.
As a further improvement of the present invention, the first foreign object calibration strategy further includes:
s21: establishing a distance coordinate system containing an actual depth value and a predicted depth value, wherein the abscissa reflects the scanning time of the laser radar and the ordinate reflects the depth value;
s22: comparing the difference value of the actual depth value and the predicted depth value at each moment, and if the continuity difference value appears and appears to tend to be horizontal in a coordinate system, calculating a foreign matter height value through a first dimension algorithm;
s23: and if the foreign matter height value is larger than the first threshold value, generating the first foreign matter information.
As a further improvement of the present invention, the image processing module further includes a second foreign object identification model, a second threshold, a second size algorithm, and a foreign object judgment policy, where the image processing module obtains the divided image information, identifies the warning area through the second foreign object identification model, and if a foreign object is identified, generates second foreign object information, where the second foreign object information reflects an actual position of the foreign object in the warning area;
the foreign matter judging strategy comprises the following steps:
s1: the second foreign matter information is acquired, the bottom position of the foreign matter outline is identified for marking, a plurality of marking points are generated, and whether the marking points are in direct contact with the ground outline in the image or not is judged; if the contact rate of the marking points and the ground contour is higher than the second threshold value, the step S2 is carried out, and if the contact rate of the marking points and the ground contour is lower than the second threshold value, the step S4 is carried out;
s2: acquiring a plurality of marking points, making a plurality of horizontal extension lines penetrating through a track in an image according to the plurality of marking points, and selecting the horizontal extension line with the largest overlapping times of the plurality of horizontal extension lines as a datum line;
s3: measuring a distance value in an image at the joint of the datum line and the two tracks, and bringing the distance value into the second dimension algorithm to generate a first depth value, wherein the first depth value reflects the measured depth value of the foreign matter;
s4: and calling the laser radar to measure the distance at the foreign object calibration point, and taking the average value of the distance measurement result as the actual depth value of the foreign object.
As a further improvement of the present invention, the foreign matter judgment policy further includes:
s31: and calling the laser radar to measure the distance of the foreign object at the standard point, taking the average value of the distance measurement result and comparing the average value with the first depth value, taking the average value as the actual depth value if the comparison result is within the error range, and re-measuring and calculating the actual depth value of the foreign object in the next frame of image if the comparison result is outside the error range.
As a further improvement of the present invention, the foreign matter judgment policy further includes:
s5: and obtaining the distance value, calculating the proportional relation between the track distance value of the position of the section and the outline size of the foreign matter, and calculating to obtain the actual ruler of the foreign matter.
The invention further improves the method, further comprises a dimension measuring module, wherein the dimension measuring module is used for obtaining the recognition result of the first foreign matter recognition model, calibrating the track position of the foreign matter if the recognition result is non-biological, calculating the proportional relation between the track distance of the section of the track position and the outline dimension of the foreign matter, and calculating the actual dimension of the foreign matter through the proportional relation and the actual width of the track.
As a further improvement of the invention, the camera module is also provided with an infrared camera, the infrared camera and the visual sensor acquire front images of the train, and the infrared camera and the images of the visual sensor are matched and fused through a heterogeneous matching algorithm.
As a further improvement of the present invention, the millimeter wave radar acquires the actual position of the foreign matter in the second foreign matter information and tracks the foreign matter in real time.
The invention further improves the train driver, and further comprises a warning module which acquires the first foreign matter information and the second foreign matter information in real time and feeds back the information to the train driver in real time.
The invention has the beneficial effects that: the invention comprises a train obstacle detection system for detecting the obstacle in the running process of the train, wherein in the obstacle recognition process of the prior art, an image is adopted by a visual sensor, then the obstacle in the image is recognized by a roll neural network and other modes, the calculated amount of the obstacle recognition mode is large, the hardware requirement is high, the recognition time is long, and the timeliness is low; and for the determination of the foreign matters in the warning area, the second foreign matter identification model is adopted to directly identify the foreign matters, and the foreign matters, especially the foreign matters which invade the track, are identified in the mode, so that the identification efficiency is greatly improved.
Compared with the prior art that the center directly measures the depth information of the foreign matters through the laser radar, the method and the device calculate the depth of the foreign matters through the width of the track in the image, overcome the defect of poor anti-interference capability of the laser radar, save the time of laser radar ranging, further improve the ranging efficiency, and have difficulty in determining the vertical plane of the corresponding part of the foreign matters and the track under the condition that the bottoms of the foreign matters are blocked by objects such as leaves, shrubs and the like, so the method and the device also directly use the laser radar for ranging.
Drawings
FIG. 1 is a flow chart of a first foreign object calibration strategy of the invention;
FIG. 2 is a flow chart of a foreign object determination strategy of the present invention;
FIG. 3 is a schematic representation of a first dimension algorithm meter of the present invention;
FIG. 4 is a schematic view of predicted depth values in a distance coordinate system according to the present invention;
FIG. 5 is a schematic view of actual depth values in a distance coordinate system according to the present invention;
fig. 6 is a schematic diagram of a fiducial line in the identification process of the present invention.
Detailed Description
The invention will now be described in further detail with reference to the drawings and examples. Wherein like parts are designated by like reference numerals. It should be noted that the words "front", "back", "left", "right", "upper" and "lower" used in the following description refer to directions in the drawings, and the words "bottom" and "top", "inner" and "outer" refer to directions toward or away from, respectively, the geometric center of a particular component.
The train obstacle detection system comprises a camera module, a foreign matter intrusion judging module, an image processing module, a laser radar and a millimeter wave radar;
the camera module is provided with a visual sensor and a region segmentation strategy, the visual sensor generates image data in real time, a visual system formed by the visual sensor is arranged at the train head, images at the front end of the train can be continuously acquired, the image data is subjected to region division through the region segmentation strategy to generate segmented image information containing an intrusion region and a warning region, the region segmentation strategy firstly extracts scene information through mode characteristics including but not limited to an image pyramid, an ORB, a SIFT and the like, a track image in the scene information is identified, then the region above the track image is judged to be the intrusion region, and the two sides of the track image are judged to be the warning region;
the foreign object intrusion judgment module is configured with a first foreign object calibration strategy and a first threshold, and the first foreign object calibration strategy comprises:
s1: acquiring segmented image information, and invoking a laser radar to sequentially scan an intrusion region along the extending direction of the intrusion region, namely, sequentially scanning the laser radar from near to far along a track when acquiring the segmented image information, so as to generate a continuous actual depth value and a continuous predicted depth value in the intrusion region, wherein the predicted depth value refers to that the measured depth value is linearly increased in the scanning process, the linear increase of the predicted depth value is consistent in each scanning process, and the actual predicted depth value refers to that the predicted depth value is consistent in a certain period of time (in the process of scanning the foreign matter) after the foreign matter is scanned in the scanning process, so that a certain difference value is generated with the predicted depth value, and the actual distance between the foreign matter and a train head can be known under the condition that the difference value occurs;
s2: comparing the continuous actual depth value with the continuous predicted depth value, wherein the comparison refers to the comparison of the actual depth value and the predicted depth value at the same moment, if the comparison result is larger than a first threshold value, the first threshold value refers to the height of an obstacle, the height of the obstacle influences whether a train can strike the obstacle or not, and meanwhile, whether the obstacle can actually influence the train or not is influenced, semantic segmentation is performed, semantic segmentation of image recognition is assisted through laser radar ranging is performed, accuracy is improved, a foreign object region is extracted, calibration is performed, first foreign object information is generated, and the first foreign object information reflects the actual position of the calibrated in an intrusion region;
the image processing module comprises a first foreign matter identification model, the image processing module acquires first foreign matter information, and the calibration part is identified and classified and checked through the first foreign matter identification model.
The invention comprises a train obstacle detection system for detecting the obstacle in the running process of the train, wherein in the obstacle recognition process of the prior art, an image is adopted by a visual sensor, then the obstacle in the image is recognized by a roll neural network and other modes, the calculated amount of the obstacle recognition mode is large, the hardware requirement is high, the recognition time is long, and the timeliness is low; and for the determination of the foreign matters in the warning area, the second foreign matter identification model is adopted to directly identify the foreign matters, and the foreign matters, especially the foreign matters which invade the track, are identified in the mode, so that the identification efficiency is greatly improved.
Specifically, the first foreign object calibration strategy further includes a scanning threshold, and in the step S1, the laser radar scans the intrusion area, and when the scanning distance reaches the scanning threshold, scanning is stopped.
Further, the foreign matter intrusion judging module further comprises a first adjusting unit, when the intrusion area is deviated, the scanning threshold is reduced to the deviation inflection point, and the foreign matter intrusion judging strategy is ensured to be only aimed at detecting the foreign matters under the straight track through the setting of the first adjusting unit.
Specifically, the foreign matter intrusion judging module is provided with a second adjusting unit, and the second adjusting unit automatically adjusts the size of the scanning threshold according to weather conditions, so that the reliability and stability of each time of scanning of the foreign matters are ensured.
Further, the weather conditions mainly reflect the influence of the weather such as rainwater, heavy fog and the like on the laser radar measurement, and the size of the visibility can be measured through the visibility detector and other instruments at the train head, so that the auxiliary second adjusting unit adjusts the scanning threshold value.
In particular, the first threshold is set to a maximum value of less than 20cm, which has the advantage that the detected height of the obstacle is in any case higher than that of the human body, and that no matter what posture the human body is on the track, the detected height can be recognized.
Specifically, the vision sensor is provided with a plurality of, sets up respectively in each position of vehicle front end, and the integration of a plurality of vision sensors plays data preprocessing work, judges the position appearance of a plurality of cameras, complements scene information, fuses the picture of taking of a plurality of vision sensors into a picture and is prior art, does not do too much in this text.
Specifically, the first foreign object identification model is trained through a database and network data to identify all possible foreign objects, and the identification modes include, but are not limited to, CNN, fasterRCNN, image pyramid, ORB, SIFT and other modes to extract obstacle information.
In one embodiment, the laser radars are provided with a plurality of groups and are divided into two groups, wherein the first group is arranged on the upper side of the train head, and the laser radars are used for assisting the semantic segmentation of the step S2 and the calibration of the foreign object region in the first foreign object calibration strategy, and are also used for the step S4 in the foreign object judgment strategy: calling a laser radar to measure the distance at a foreign object calibration point and S31: calling a laser radar to measure the distance at a foreign object calibration point; the group of radars can mainly identify foreign matters as directly as possible, and therefore, the radars need to be arranged on the upper side of the train head; the second group is arranged at the lower side in the train head, and the group of laser radars are mainly used for assisting the invoking laser radars of the step S1 in the first foreign matter calibration strategy to sequentially scan the intrusion region along the extending direction of the intrusion region to generate continuous actual depth values and continuous predicted depth values in the intrusion region, because the foreign matters need to be scanned as much as possible in the process of scanning the intrusion region, when the position of the laser radars is arranged at the lower position, the larger the area of the intrusion region is shielded by the foreign matters, the longer the scanning time of the laser radars is, so that the longer the horizontal line length of the continuity difference value in the distance coordinate system is, the foreign matter height values can be calculated more accurately, and the problem that the accuracy of the measurement result is reduced because the scanning speed of the laser radars is too fast is solved.
In one embodiment, the first foreign object calibration strategy further comprises:
s21: establishing a distance coordinate system containing an actual depth value and a predicted depth value, wherein the abscissa reflects the scanning time of the laser radar and the ordinate reflects the depth value;
s22: comparing the difference value of the actual depth value and the predicted depth value at each moment, if the continuity difference value appears and the continuity difference value appears to be in a trend level in a coordinate system, wherein the continuity difference value appears, namely that the actual depth value and the predicted depth value appear to split in the trend in the coordinate system, and the actual depth value does not increase along with the increase of the moment, so the actual depth value appears to be in a level in the coordinate system, and the situation shows that at the moment, the obstacle blocks the track, and the foreign matter height value is calculated through a first size algorithm;
s23: if the foreign matter height value is greater than the first threshold value, first foreign matter information is generated.
In one embodiment, as shown in the accompanying drawings, the first algorithm is:
wherein H is the actual height of the obstacle, L1 is the end value of the horizontal extension section of the actual depth value in the coordinate system, L2 is the value of the next time the end of the horizontal extension section of the actual depth value appears, H 1 Height set for the sensor.
In one embodiment, the image processing module further includes a second foreign object identification model, a second threshold, a second size algorithm, and a foreign object determination policy, the image processing module obtains the divided image information, identifies the warning area through the second foreign object identification model, and if a foreign object is identified, generates second foreign object information, where the second foreign object information reflects an actual position of the foreign object in the warning area;
the foreign matter judgment strategy comprises:
s1: acquiring second foreign matter information, identifying the bottom position of the foreign matter outline, marking and generating a plurality of marking points, and judging whether the marking points are in direct contact with the ground outline in the image or not; the bottom of the outline is marked with a plurality of points to judge whether the outline is in direct contact with the ground, so that misjudgment in the judging process is effectively prevented, the success rate of point position judgment is improved, if the contact rate of a plurality of marked points and the outline of the ground is higher than a second threshold value, the step S2 is carried out, and if the contact rate of a plurality of marked points and the outline of the ground is lower than the second threshold value, the step S4 is carried out;
s2: acquiring a plurality of marking points, making a plurality of horizontal extension lines penetrating through the tracks in the image according to the plurality of marking points, wherein the tracks in the synthesized image extend forwards perpendicular to the headstock, making the horizontal extension lines of the marking points to find out that foreign matters are positioned on the same vertical plane with the tracks, selecting the horizontal extension line with the largest overlapping times of the plurality of horizontal extension lines as a datum line, and selecting the horizontal extension line with the largest overlapping times as the datum line because the plurality of selected marking points are not necessarily positioned on the same horizontal plane due to the irregularity of the contour, so that the measurement result is the most accurate;
s3: measuring the distance value of the joint of the datum line and the two tracks in the image, and bringing the distance value into a second dimension algorithm to generate a first depth value, wherein the first depth value reflects the measured depth value of the foreign matter, and the second dimension algorithm can calculate the first depth value by multiplying the distance value in the image by the actual distance of the tracks because the distance between the tracks is constant;
s4: and (5) calling the laser radar to measure the distance at the foreign object calibration point, and taking the average value of the distance measurement result as the actual depth value of the foreign object.
Compared with the prior art that the center directly measures the depth information of the foreign matters through the laser radar, the method and the device calculate the depth of the foreign matters through the width of the track in the image, overcome the defect of poor anti-interference capability of the laser radar, save the time of laser radar ranging, further improve the ranging efficiency, and have difficulty in determining the vertical plane of the corresponding part of the foreign matters and the track under the condition that the bottoms of the foreign matters are blocked by objects such as leaves, shrubs and the like, so the method and the device also directly use the laser radar for ranging.
In one embodiment, the second foreign object identification model is trained through database and network data, and the identification objects mainly comprise automatically movable obstacles such as living beings, vehicles and the like, and the identification modes comprise, but are not limited to, CNN, fasterRCNN, image pyramid, ORB, SIFT and the like, and obstacle information is extracted.
In one embodiment, the second threshold is a percentage, and may be set by self-setting, and the default second threshold is eighty-five percent, that is, in the identifying process of a plurality of marking points, the identifying result of each marking point cannot be guaranteed to be the same, that is, it is possible that part of the marking points overlap with the ground contour, and part of the marking points are blocked by objects such as leaves, so that the second threshold is set in order to guarantee the accuracy of the foreign matter judging strategy.
In one embodiment, the foreign object determination policy further includes:
s31: and calling the laser radar to measure the distance at the foreign object calibration point, taking the average value of the distance measurement result and comparing the average value with the first depth value, taking the average value as the actual depth value if the comparison result is within the error range, and re-measuring and calculating the actual depth value of the foreign object in the next frame of image if the comparison result is outside the error range.
Through step S31 as a verification step, the laser radar ranging numerical value and the numerical value which is measured and converted through the track in the image can be compared and verified, the accuracy of the numerical value result can be ensured, and the deviation of one ranging result is prevented.
In one embodiment, the foreign object determination policy further includes:
s5: and obtaining a distance value, calculating the proportional relation between the track distance value of the section position and the outline size of the foreign matter, and calculating to obtain the actual ruler of the foreign matter.
According to the method, after the distance value is obtained, the size of the foreign matter can be measured by directly utilizing the track image width corresponding to the datum line, namely, the distance of the track is measured as a measuring scale, the size of the foreign matter can be rapidly judged by measuring the distance, meanwhile, the image with the datum line is displayed through the display equipment, and a driver can intuitively know the size of the foreign matter.
In one embodiment, the device further comprises a dimension measurement module, a first foreign matter identification model is obtained, if the identification result is non-biological, the track position where the foreign matter is located is calibrated, the proportional relation between the track distance of the position and the outline dimension of the foreign matter is calculated, and the actual dimension of the foreign matter is calculated through the proportional relation and the actual width of the track.
In the driving process of the train, the processing results of different foreign matters are different, the size of the non-biological foreign matters need to be measured after the types are identified through the arrangement of the size measuring module, the foreign matters which are not affected by the collision of the vehicle can be reported without the warning module, and meanwhile, the size measuring module calculates the size of the foreign matters through the proportional relation between the track spacing of the section position and the oversized foreign matter wheels.
In one embodiment, the camera module is further provided with an infrared camera, the infrared camera and the vision sensor acquire front images of the train, the images of the infrared camera and the vision sensor are matched and fused through a heterogeneous matching algorithm, the infrared camera realizes functions of far-end gain and near-end inhibition through a brightness balancing algorithm, and the far-end and near-end obstacle contours can be identified and accurately identified; the more obvious the generated image features are, the accuracy of image recognition and the accuracy of a laser radar measurement result are effectively improved, the infrared image provides coarse-granularity information for the obstacle, the vision sensor provides fine-granularity information, and high-robustness obstacle detection and recognition are realized through a mode of fusion of various image information.
In one embodiment, the millimeter wave radar obtains the actual position of the foreign matter in the second foreign matter information and tracks the foreign matter in real time, the detection result of the millimeter wave radar and the detection result of the image are summarized and analyzed through the pre-pose calibration and the real-time analysis of the depth of the scene by the visible light camera, the most accurate detection result is obtained through the maximum likelihood estimation of various sensor information, the millimeter wave radar can realize autonomous target detection, tracking, distance measurement, speed measurement and acceleration measurement, no other sensors are needed to participate in information extraction, but the millimeter wave radar result is only sensitive to moving metal objects (the tracking effect on vehicles is best) in order to remove the interference of the environment. The millimeter wave radar performs semantic combination with a vision and laser radar result after detecting an obstacle target, and adds the motion situation of the detected object to an output result, and a specific millimeter wave radar tracking algorithm is not described in detail in the prior art.
In one embodiment, the train driver monitoring system further comprises a warning module, wherein the warning module acquires the first foreign matter information and the second foreign matter information in real time and feeds back the first foreign matter information and the second foreign matter information to the train driver in real time.
Specifically, the warning module prompts through a display, a loudspeaker and the like on the train.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.
Claims (9)
1. A train obstacle detection system, characterized by: the system comprises a camera module, a foreign matter intrusion judging module, an image processing module, a laser radar and a millimeter wave radar;
the camera module is configured with a visual sensor and a region segmentation strategy, the visual sensor generates image data in real time, the image data is subjected to region segmentation through the region segmentation strategy, and segmented image information containing an intrusion region and a warning region is generated;
the foreign object intrusion judgment module is configured with a first foreign object calibration strategy and a first threshold, and the first foreign object calibration strategy comprises:
s1: acquiring the segmentation image information, and calling the laser radar to sequentially scan the intrusion region along the extending direction of the intrusion region to generate a continuous actual depth value and a continuous predicted depth value in the intrusion region;
s2: comparing the continuous actual depth value with the continuous predicted depth value, if the comparison result is larger than a first threshold value, performing semantic segmentation, extracting a foreign object region, and calibrating to generate first foreign object information, wherein the first foreign object information reflects the actual position of the invasive region in the calibration;
the image processing module comprises a first foreign matter identification model, the image processing module acquires the first foreign matter information, and the calibration part is identified and classified and checked through the first foreign matter identification model.
2. A train obstacle detection system according to claim 1, wherein: the first foreign object calibration strategy further comprises:
s21: establishing a distance coordinate system containing an actual depth value and a predicted depth value, wherein the abscissa reflects the scanning time of the laser radar and the ordinate reflects the depth value;
s22: comparing the difference value of the actual depth value and the predicted depth value at each moment, and if the continuity difference value appears and appears to tend to be horizontal in a coordinate system, calculating a foreign matter height value through a first dimension algorithm;
s23: and if the foreign matter height value is larger than the first threshold value, generating the first foreign matter information.
3. A train obstacle detection system according to claim 1, wherein: the image processing module further comprises a second foreign matter identification model, a second threshold value, a second size algorithm and a foreign matter judging strategy, the image processing module acquires the split image information, the warning area is identified through the second foreign matter identification model, if the foreign matter is identified, second foreign matter information is generated, and the second foreign matter information reflects the actual position of the foreign matter in the warning area;
the foreign matter judging strategy comprises the following steps:
s1: the second foreign matter information is acquired, the bottom position of the foreign matter outline is identified for marking, a plurality of marking points are generated, and whether the marking points are in direct contact with the ground outline in the image or not is judged; if the contact rate of the marking points and the ground contour is higher than the second threshold value, the step S2 is carried out, and if the contact rate of the marking points and the ground contour is lower than the second threshold value, the step S4 is carried out;
s2: acquiring a plurality of marking points, making a plurality of horizontal extension lines penetrating through a track in an image according to the plurality of marking points, and selecting the horizontal extension line with the largest overlapping times of the plurality of horizontal extension lines as a datum line;
s3: measuring a distance value in an image at the joint of the datum line and the two tracks, and bringing the distance value into the second dimension algorithm to generate a first depth value, wherein the first depth value reflects the measured depth value of the foreign matter;
s4: and calling the laser radar to measure the distance at the foreign object calibration point, and taking the average value of the distance measurement result as the actual depth value of the foreign object.
4. A train obstacle detection system according to claim 3, wherein: the foreign matter judgment policy further includes:
s31: and calling the laser radar to measure the distance of the foreign object at the standard point, taking the average value of the distance measurement result and comparing the average value with the first depth value, taking the average value as the actual depth value if the comparison result is within the error range, and re-measuring and calculating the actual depth value of the foreign object in the next frame of image if the comparison result is outside the error range.
5. A train obstacle detection system according to claim 3, wherein: the foreign matter judgment policy further includes:
s5: and obtaining the distance value, calculating the proportional relation between the track distance value of the position of the section and the outline size of the foreign matter, and calculating to obtain the actual ruler of the foreign matter.
6. A train obstacle detection system according to claim 3, wherein: the device further comprises a dimension measurement module, wherein the dimension measurement module is used for acquiring the recognition result of the first foreign matter recognition model, calibrating the track position where the foreign matters are located if the recognition result is non-living, calculating the proportional relation between the track space at the position and the outline dimension of the foreign matters, and calculating the actual dimension of the foreign matters through the proportional relation and the actual width of the track.
7. A train obstacle detection system according to claim 1, wherein: the camera module is also provided with an infrared camera, the infrared camera and the visual sensor acquire images in front of the train, and the images of the infrared camera and the visual sensor are matched and fused through a heterogeneous matching algorithm.
8. A train obstacle detection system according to claim 1, wherein: the millimeter wave radar acquires the actual position of the foreign matter in the second foreign matter information and tracks the foreign matter in real time.
9. A train obstacle detection system according to claim 1, wherein: the train driver warning device further comprises a warning module, wherein the warning module acquires the first foreign matter information and the second foreign matter information in real time and feeds the first foreign matter information and the second foreign matter information back to the train driver in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310903980.5A CN117011828A (en) | 2023-07-21 | 2023-07-21 | Train obstacle detecting system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310903980.5A CN117011828A (en) | 2023-07-21 | 2023-07-21 | Train obstacle detecting system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117011828A true CN117011828A (en) | 2023-11-07 |
Family
ID=88575673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310903980.5A Pending CN117011828A (en) | 2023-07-21 | 2023-07-21 | Train obstacle detecting system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117011828A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119935017A (en) * | 2025-04-08 | 2025-05-06 | 中数智科(杭州)科技有限公司 | A sensor position calibration system for limit recognition |
-
2023
- 2023-07-21 CN CN202310903980.5A patent/CN117011828A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119935017A (en) * | 2025-04-08 | 2025-05-06 | 中数智科(杭州)科技有限公司 | A sensor position calibration system for limit recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111198371B (en) | Forward-looking obstacle detection system | |
| KR101925293B1 (en) | The vehicle detecting system by converging radar and image | |
| US6130706A (en) | Process for determining vehicle dynamics | |
| US7046822B1 (en) | Method of detecting objects within a wide range of a road vehicle | |
| JP3733875B2 (en) | Road white line recognition device | |
| US10501059B2 (en) | Stereo camera device | |
| US9349057B2 (en) | Three-dimensional object detection device | |
| CN110609274B (en) | Distance measurement method, device and system | |
| US6477260B1 (en) | Position measuring apparatus using a pair of electronic cameras | |
| US20060111841A1 (en) | Method and apparatus for obstacle avoidance with camera vision | |
| US20040178945A1 (en) | Object location system for a road vehicle | |
| US20070211919A1 (en) | Vehicle surroundings monitoring apparatus | |
| KR101898051B1 (en) | Multilane vehicle speed detecting system | |
| US5835880A (en) | Apparatus and method for vehicle following with dynamic feature recognition | |
| CN107796373B (en) | Distance measurement method based on monocular vision of front vehicle driven by lane plane geometric model | |
| Lion et al. | Smart speed bump detection and estimation with kinect | |
| CN110659552A (en) | Tramcar obstacle detection and alarm method | |
| Adamshuk et al. | On the applicability of inverse perspective mapping for the forward distance estimation based on the HSV colormap | |
| CN113353071B (en) | Narrow area intersection vehicle safety auxiliary method and system based on deep learning | |
| CN117471463B (en) | Obstacle detection method based on 4D radar and image recognition fusion | |
| JP2005157731A (en) | Lane recognition device and lane recognition method | |
| CN117011828A (en) | Train obstacle detecting system | |
| CN111998780B (en) | Target ranging method, device and system | |
| US7885430B2 (en) | Automotive environment monitoring device, vehicle with the automotive environment monitoring device, and automotive environment monitoring program | |
| KR102062579B1 (en) | Vehicle license-plate recognition system that recognition of Vehicle license-plate damaged by shadow and light reflection through the correction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |