KR102709531B1 - Video analysis based road risk control device and method thereof - Google Patents

Abstract

One aspect of the present invention provides an image analysis-based road hazard control device. The image analysis-based road hazard control device may include at least one processor and a memory storing instructions for instructing the at least one processor to perform at least one step. The at least one step may include a step of receiving road information from an external server, a step of analyzing a road image to obtain road image information, a step of calculating a structure information score, an environment information score, and a traffic information score by mutually combining the road information, and a step of calculating a road information score from a weighted sum of the calculated structure information score, environment information score, and traffic information score, a step of calculating a surface condition information score and an obstacle information score by mutually combining necessary road image information, and a step of calculating a road image information score from a weighted sum of the calculated surface condition information score and obstacle information score, a step of calculating a road hazard factor score from the weighted sum of the calculated road information score and the road image information score, and a step of calculating an adjusted communication number by multiplying a predetermined basic communication number by the calculated road hazard factor score.

Description

{VIDEO ANALYSIS BASED ROAD RISK CONTROL DEVICE AND METHOD THEREOF}

The present invention relates to an image analysis-based road hazard control device and an operating method thereof, and more specifically, to an image analysis-based road hazard control device and an operating method thereof that analyzes and controls hazardous materials in a road section by analyzing dynamically collected images.

Concerns about and demands for safety in the road traffic sector are continuously increasing.

In the past, most of the information for judging road safety was based on communication information centered on traffic speed. However, in the field of road traffic, safety requires technological research that can resolve unstable situations that drivers can actually experience while driving, such as road weather and road hazard information.

Conventional technology collects road safety information by installing equipment or systems fixed at specific points on the roadside and provides selective information to each driver according to his/her choice through a centralized server.

This has limitations in that it is difficult to grasp the actual driving risk information according to the road section in a timely and appropriate manner, and there is a problem that it does not provide drivers with real-time changing road conditions, and does not provide sufficient efficiency for drivers' road traffic safety due to the need to go through a central server.

Accordingly, research is required on a road risk control method based on image analysis to directly collect and analyze data from vehicles driving on the road through devices attached to individual vehicles driving on the road and provide direct safety information for vehicle operation.

Domestic registration patent No. 10-2264392

The purpose of the present invention to solve the above problems is to provide an image analysis-based road hazard control device and its operating method.

One aspect of the present invention to achieve the above object provides an image analysis-based road hazard control device and an operating method thereof.

A road hazard control device based on image analysis may include at least one processor and a memory storing instructions that instruct the at least one processor to perform at least one step.

At least one of the steps may include: receiving road information from an external server; analyzing a road image to obtain road image information; calculating a structure information score, an environment information score and a traffic information score by mutually combining the road information; calculating a road information score from a weighted sum of the calculated structure information score, environment information score and traffic information score; calculating a surface condition information score and an obstacle information score by mutually combining necessary road image information; calculating a road image information score from a weighted sum of the calculated surface condition information score and obstacle information score; calculating a road hazard factor score from the weighted sum of the calculated road information score and the road image information score; and calculating an adjusted communication count by multiplying a predetermined basic communication count by the calculated road hazard factor score.

The step of calculating the road information score can calculate the road information score by calculating the structural information score from the sum of the reciprocal of the road width, the reciprocal of the road slope, and the reciprocal of the curvature radius, calculating the environmental information score from the sum of the values calculated by the reciprocal of the road surface temperature, the square of the reciprocal of the visibility, and the product of the amount of precipitation and the wind speed, and calculating the traffic information score from the sum of the sigmoid function values for the normalized values of the traffic volume, the average vehicle speed, and the accident history.

The step of calculating the road image information score may calculate the surface condition information score from the weighted sum of potholes, cracks, and lane markings, and may calculate the obstacle information score from the sum of the sigmoid function values for the normalized values of fallen objects, stopped vehicles, and construction barriers.

When using the image analysis-based road hazard control device and its operation method according to the present invention as described above, road safety and management efficiency can be improved by analyzing and controlling road hazards in real time by analyzing dynamically collected images.

The effects obtainable from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.

The above and other aspects, features and advantages of certain preferred embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings.
FIG. 1 is an exemplary diagram showing the operating environment of a road hazard control device based on image analysis according to one embodiment of the present invention.
Figure 2 is an exemplary diagram illustrating an example of dynamic image collection of a road hazard control device based on image analysis.
Figure 3 is an exemplary diagram illustrating an example of a road hazard control device based on image analysis.
Figure 4 is an example diagram illustrating an example of using image analysis results of an image analysis-based road hazard control device.
Figure 5 is an exemplary diagram illustrating another example of a road hazard control device based on image analysis.
Fig. 6 is a hardware configuration diagram for a road hazard control device based on image analysis according to Fig. 1.
It should be noted that throughout the above drawings, like reference numerals are used to illustrate identical or similar elements, features and structures.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

In describing the embodiments, descriptions of technical contents that are well known in the technical field to which the present invention belongs and are not directly related to the present invention will be omitted. This is to convey the gist of the present invention more clearly without obscuring it by omitting unnecessary descriptions.

For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically illustrated. In addition, the size of each component does not entirely reflect the actual size. The same or corresponding components in each drawing are given the same reference numbers.

The advantages and features of the present invention, and the methods for achieving them, will become clear with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and the present embodiments are provided only to make the disclosure of the present invention complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions. These computer program instructions can be loaded onto a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in the flow diagram block(s). These computer program instructions can also be stored in a computer-available or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in the flow diagram block(s). Since the computer program instructions may also be installed on a computer or other programmable data processing apparatus, a series of operational steps may be performed on the computer or other programmable data processing apparatus to produce a computer-executable process, so that the instructions executing the computer or other programmable data processing apparatus may also provide steps for executing the functions described in the flowchart block(s).

Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a particular logical function(s). It should also be noted that in some alternative implementation examples, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may in fact be performed substantially concurrently, or the blocks may sometimes be performed in reverse order, depending on the functionality they perform.

Here, the term '~ part' used in the present embodiment means software or hardware components such as FPGA (field-programmable gate array) or ASIC (application specific integrated circuit), and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be in an addressable storage medium and may be configured to reproduce one or more processors. Accordingly, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided in the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. Additionally, the components and '~parts' may be implemented to regenerate one or more CPUs within the device or secure multimedia card.

In specifically describing embodiments of the present invention, examples of specific systems will be mainly targeted, but the main points claimed in this specification can be applied to other communication systems and services having similar technical backgrounds without significantly departing from the scope disclosed in this specification, and this can be done at the discretion of a person skilled in the relevant technical field.

FIG. 1 is an exemplary diagram showing the operating environment of a road hazard control device based on image analysis according to one embodiment of the present invention.

Referring to FIG. 1, an image analysis-based road hazard control device (100) (hereinafter referred to as 'device (100)') can provide image analysis results through a user terminal (10).

Here, the user terminal (10) may include a communicable desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, a smart glass, an e-book reader, a portable multimedia player (PMP), a portable game console, a navigation device, a digital camera, a digital multimedia broadcasting (DMB) player, a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, a PDA (Personal Digital Assistant), etc.

In addition, the device (100) can be connected to an external integrated control server (110) via a wired or wireless network to transmit the image analysis results of the device (100).

Here, the wired/wireless network may include various communication methods such as, but is not limited to, Bluetooth communication, BLE (Bluetooth Low Energy) communication, Near Field Communication (NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, WFD (Wi-Fi Direct) communication, UWB (ultra-wideband) communication, Ant+ communication, WIFI communication, and RFID (Radio Frequency Identification) communication.

In addition, the device (100) may include at least one sensor for image recognition. For example, the sensor may include, but is not limited to, a general camera, a stereo camera, a 3D camera, a thermal imaging camera, a night vision camera, a wide-angle camera, an ultra-high-resolution camera, an infrared camera, a LiDAR, a radar, an ultrasonic sensor, an infrared sensor, a GPS sensor, a gyroscope, an accelerometer, an acoustic sensor, an environmental sensor (e.g., temperature, humidity, barometric pressure), an IMU (inertial measurement unit), a microphone, a vision sensor, a magnetometer, an optical sensor, a Doppler radar, a magnetic field sensor, and a laser range finder.

Meanwhile, the device (100) can receive road information from an external server and obtain road image information from at least one sensor for image recognition.

Here, road information may include road width, number of lanes, lane width, road slope, curvature radius, road surface type, temperature, humidity, visibility, road surface temperature, precipitation, wind speed, traffic volume, average vehicle speed, accident history, congestion level, and road construction zones.

Additionally, the road image information herein may include potholes, cracks, lane markings, debris, water accumulation, skid marks, manhole covers, fallen objects, stopped vehicles, construction barriers, crosswalks, speed bumps, and signs, as information obtained from the received road image.

The device (100) can classify road information and road image information into at least one category according to at least one attribute.

Next, the device (100) can combine road information classified into each category to calculate a road information score.

When the device (100) acquires road image information, it can classify the road image information into at least one category based on at least one attribute.

Next, the device (100) can combine road image information classified into each category to calculate a road image information score.

Meanwhile, the device (100) can calculate a road risk factor score by mutually combining the calculated road information score and road image information score.

Next, the device (100) can calculate the number of adjusted communications and calculate the adjusted communication bandwidth using the calculated road risk factor score.

Through this, the device (100) can efficiently control road hazards by detecting and controlling road hazards in real time.

Figure 2 is an exemplary diagram illustrating an example of dynamic image collection of a road hazard control device based on image analysis.

For example, referring to FIGS. 1 and 2, a road (200) may be operated by at least one vehicle (210, 220, 230), and each vehicle may be equipped with a device (100).

Here, the road (200) may include, but is not limited to, a highway, a national road, a local road, a bicycle road, a pedestrian road, a mountain road, a farm road, a coastal road, a tunnel, a bridge, an intersection, a bypass, a one-way road, a complex road, an overpass, an underpass, and a walking path, and may include all sections in which a road hazard control device based on image analysis is installed and can move.

In addition, vehicles are vehicles intended for the transport of cargo and passengers, and may include, but are not limited to, buses and freight cars, and may include passenger cars, bicycles, motorcycles, and special purpose vehicles.

The device (100) is mounted on each vehicle and can detect road hazards on the road (200). Here, the hazards are defined as potholes, black ice, fallen objects, animals, people, ice, snow, rain, oil, flooding, road construction, obstacles, street trees, broken streetlights, broken traffic lights, traffic accident debris, speed bumps, road surface conditions, obstructions to the view, damaged road signs, sharp curves, steep slopes, and road icing, which interfere with normal vehicle operation or are abnormal conditions in which the road (200) cannot perform its original purpose, and may include all unstable situations that a driver may experience while driving the vehicle.

Meanwhile, the device (100) can determine road hazards by analyzing images acquired while driving on a road (200).

That is, the device (100) can be mounted on at least one vehicle driving on a road (200), and can increase the reliability of road hazard detection and judgment through independent image analysis and road hazard detection and judgment.

For example, the device (100) can transmit the results of image analysis and road hazard detection and judgment to the integrated control server (110), and the device (100) can receive statistical information on the results of image analysis and road hazard detection and judgment via the integrated control server (110).

At this time, the device (100) can modify the image analysis and road hazard detection and judgment results based on the statistical data of the received image analysis and road hazard detection and judgment results.

For example, vehicles (210, 220) are driving in the same direction on the same road (200). At this time, the image information and analysis results obtained from the vehicles (210, 220) and the detection of road hazards may be the same, but the results may be derived differently depending on independent analysis.

At this time, each device (100) of the vehicle (210, 220) can transmit the results of image analysis and road hazard detection and judgment to the integrated control server (110). Meanwhile, the integrated control server (110) can receive the results of image analysis and road hazard detection and judgment for multiple vehicles that passed the road (200) within a predetermined time range (e.g., 24 hours) and can generate statistical information on the results of road hazard detection and judgment.

Next, each device (100) of the vehicle (210, 220) can receive statistical information on the road hazard detection and judgment results generated from the integrated control server (110) and modify the road hazard detection and judgment results.

Accordingly, the device (100) has low latency through its own image analysis and road risk judgment, can communicate with the integrated control server (110) using a narrow bandwidth communication network, and can perform efficient road risk analysis with low data processing capacity.

Figure 3 is an exemplary diagram illustrating an example of a road hazard control device based on image analysis.

Referring to FIGS. 1 and 3, the device (100) may include an input unit (310), a control unit (320), and an output unit (330).

At this time, the input unit (310) may include at least one sensor for image acquisition, for example, a camera (311) for CCD/COMS image input and a thermal imaging camera (312) for thermal image input.

The control unit (320) can process an image acquired through the input unit (310), and for example, can include a CCD/COMS image processing unit (321) for CCD/COMS image processing and a thermal image processing unit (322) for thermal image processing.

At this time, the image processing may include preprocessing for image analysis, and may include, but is not limited to, noise removal, histogram equalization, edge detection, background removal, grayscale conversion, image/image resizing, color correction, region of interest (ROI) setting, contour detection, feature extraction, and image segmentation.

In addition, the image analysis unit (323) can receive a preprocessed image and analyze the image. For example, the image analysis unit (323) can perform image analysis for object recognition through motion detection, segmentation, feature matching, and pattern recognition in the image. In addition, the present invention is not limited thereto, and the image analysis unit (323) can analyze the road situation using a road situation learning model, and the road situation learning model can be an artificial intelligence-based machine learning or deep learning model, and for example, an artificial intelligence learning model including supervised learning, unsupervised learning, reinforcement learning, CNNs (Convolutional Neural Networks), RNNs (Recurrent Neural Networks), LSTMs (Long Short-Term Memory Networks), Natural Language Processing (NLP), Decision Support Systems (DSS), and predictive analysis can be used.

The image object identification/recognition unit (324) can identify and recognize an object from the analyzed image. For example, the image object identification/recognition unit (324) can identify an object from the analyzed image. At this time, the object identification can be identified from a segmentation area or a region of interest (ROI) of the image. In addition, the image object identification/recognition unit (324) can detect an outline and generate a bounding box for object identification. For example, an object can be detected by generating a bounding box for a detected outline that matches the characteristics of a road hazard (e.g., a pothole).

Next, the image object identification/recognition unit (324) can recognize the detected object and classify it as a road hazard. At this time, the classification of the road hazard element can use a deep learning-based classifier such as SVM (Support vector machine), Random Forest or CNN (Convolutional Neural Network), but is not limited thereto, and a pre-trained deep learning model such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) or Faster R-CNN can be used.

The output unit (330) can output recognized road hazards. For example, the output unit (330) can directly output road hazards through the device (100) via the display unit (331), and can transmit the road hazard detection results to an external server via the network unit (332).

For example, the network unit (332) can transmit the road hazard element detection results to the integrated control server (110) via TCP/IP wired/wireless communication using a security protocol.

Figure 4 is an example diagram illustrating an example of using image analysis results of an image analysis-based road hazard control device.

Referring to FIGS. 1 and 4, the device (100) can transmit an image or a road hazard detection result to the integrated control server (110). The integrated control server (110) can store the image and road hazard detection result received from the device (100) and transmit the stored image and road hazard detection result to the data sharing platform (400).

At this time, the data sharing platform (400) may include, but is not limited to, an image sharing platform, a road information sharing platform, and a map platform. The data sharing platform (400) that receives an image or a road hazard detection result from the integrated control server (110) may receive road geometry data, reproduction data, bridge data, static traffic volume, road drawing data, and road information from the public road data management system (410) to generate integrated data. At this time, the data sharing platform (400) may combine the received image or road hazard detection result with the road information to generate detailed road situation data for each road and share the generated road situation data.

For example, the data sharing platform (400) may provide road condition data generated to a private sector service or a public sector service, thereby enabling the use of images acquired through the device (100) and detected road hazard information.

Through this, the device (100) can share road condition information by providing road condition data to the data sharing platform (400) via the integrated control server (110). In addition, without being limited thereto, the device (100) can be directly connected to the data sharing platform (400) via a wired or wireless network to share video and road hazard detection results.

Figure 5 is an exemplary diagram illustrating another example of a road hazard control device based on image analysis.

Referring to FIG. 5, the image analysis-based road hazard control device (500) (hereinafter referred to as 'device (500)') may include a road information analysis unit (510), a road image analysis unit (520), and a road hazard analysis unit (530).

The road information analysis unit (510) can receive road information from an external server according to user input road information or a user request.

Here, road information may include road width, number of lanes, lane width, road slope, curvature radius, road surface type, temperature, humidity, visibility, road surface temperature, precipitation, wind speed, traffic volume, average vehicle speed, accident history, congestion level, and road construction zones.

When receiving road information, the road information analysis unit (510) can classify the received road information into at least one category based on at least one attribute.

For example, the road information analysis unit (510) can classify road width, number of lanes, lane width, road slope, curvature radius, and road surface type as structural information, classify temperature, humidity, visibility, road surface temperature, precipitation, and wind speed as environmental information, and classify traffic volume, average vehicle speed, accident history, congestion level, and road construction zone as traffic information.

The road information analysis unit (510) classifies road information classified into structural information, environmental information, and traffic information into each category and manages them, thereby enabling real-time monitoring of road information and early identification and management of any deviations from the management pattern. In addition, the convenience of information management can be increased by managing the index of information classified into each category.

Meanwhile, classification of road information categories and calculation of road information scores can use artificial intelligence-based machine learning or deep learning models, and for example, artificial intelligence learning models including supervised learning, unsupervised learning, reinforcement learning, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), natural language processing (NLP), decision support systems (DSS), and predictive analysis can be used.

In addition, the road information analysis unit (510) can monitor information classified into each category in real time and generate a notification (e.g., sound, vibration, light, etc.) for warning when a predetermined deviation range (e.g., 20%) is exceeded.

Next, the road information analysis unit (510) can calculate a road information score by mutually combining road information classified into each category.

For example, the road information analysis unit (510) can calculate a structural information score as in the mathematical formula below.

Here, the road width refers to the total road width of the same direction lane, and when the total width of a four-lane road is 16 m, the road width can be 16/2 = 8 m. As an example, the road width can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. As an example, if the minimum road width is 3 m, the maximum road width is 12 m, and the actual road width is 6 m, the road width can be normalized to 0.3. In other words, the narrower the road, the greater the road structural risk, and the narrower the road, the lower the vehicle maneuverability and the higher the possibility of a collision.

In addition, the road gradient indicates the downward (-) or upward (+) slope of the road relative to the horizontal, and the value can be used as positive (+) regardless of whether it is an uphill or downhill. The steeper the road gradient, the more it burdens the stability and controllability of the vehicle, and the greater the risk of road structural hazards.

For example, a road slope can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum road slope is 0 degrees, the maximum road slope is 20 degrees, and the actual road slope is 5 degrees, the road slope can be normalized to 0.25.

The radius of curvature is the radius of an imaginary circle about the curvature of a specific point on the road, and can be an indicator of how sharp the road curve is. In other words, a smaller radius of curvature means a sharper road curve, while a larger radius of curvature means a gentler road curve.

For example, the curvature radius can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum curvature radius is 10 m, the maximum curvature radius is 500 m, and the actual curvature radius is 50 m, the curvature radius can be normalized to 0.08.

For example, if the normalized road width, road slope, and curvature radius are 0.3, 0.25, and 0.08, respectively, the structural information score is calculated as follows.

The generated structural information score can be generated in real time according to the movement of the vehicle at a specific location on each road, and as an indicator of the risk according to each road structural element, a high structural information score can mean that the risk of the road according to the structural information at that specific location is high. Through this, the road information analysis unit (510) can independently determine road risk factors by utilizing both 'road information' and 'road image information'.

Meanwhile, the structural information score can be normalized to 0 or 1 based on the minimum and maximum structural information scores determined in advance. Through this, the calculation of the road information score described below can be balanced with other indices.

The road information analysis unit (510) can calculate the environmental information score as shown in the mathematical formula below.

Here, the road surface temperature can be a factor that increases or decreases the physical condition of the road, such as freezing (black ice) or overheating of the road surface, and the friction between the road and the vehicle tires.

For example, the road surface temperature can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum road surface temperature is -20 degrees, the maximum road surface temperature is 40 degrees, the actual road surface temperature is 30 degrees, and the ideal road surface temperature is 25 degrees, the road surface temperature can be normalized to 0.92.

Visibility can be derived from images captured by a visibility sensor or camera, and can be measured in meters. That is, visibility can be measured by a receiver that detects the amount of light that reaches the air after emitting light, such as a laser or LED, over a fixed distance.

For example, visibility can be normalized from 0 to 1 via min-max normalization over a range of predetermined minimum and maximum values. For example, if the minimum visibility is 0, the maximum visibility is 1000, and the actual visibility is 200, the visibility can be normalized to 0.2.

Precipitation is a measure of the amount of precipitation in the form of rain, snow, or other forms, and can be measured in mm/hour.

For example, precipitation can be normalized to 0 to 1 via min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum precipitation is 0, the maximum precipitation is 50 mm/hour, and the actual precipitation is 10 mm/hour, the precipitation can be normalized to 0.2.

Wind speed is a value measured by an anemometer, and can be measured in km/hour. For example, wind speed can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum wind speed is 0, the maximum wind speed is 100 km/hour, and the actual wind speed is 20 km/hour, the wind speed can be normalized to 0.2.

For example, if the normalized values of the produced or measured road surface temperature, visibility, precipitation, and wind speed are 0.92, 0.2, 0.2, and 0.2, respectively, the environmental information score is calculated as follows.

The generated environmental information score can be generated in real time according to the movement of the vehicle at a specific location on each road, and as an indicator of the risk according to the environmental factors of each road, a high environmental information score can mean that the risk of the road according to the environmental information at the specific location is high. Through this, the road information analysis unit (510) can independently determine the road risk factors by utilizing both 'road information' and 'road image information'.

Meanwhile, the environmental information score can be normalized to 0 or 1 based on the minimum and maximum environmental information scores set in advance. Through this, the calculation of the road information score described below can be balanced with other indicators.

Additionally, the road information analysis unit (510) can calculate traffic information scores as shown in the mathematical formula below.

Here, is a nonlinear function that outputs the input value as a value between 0 and 1, and is a differentiable continuous function. This allows the traffic information score to reduce the influence of extreme values and ensure a more balanced contribution to the overall index.

Here, the traffic volume can be calculated by measuring the traffic volume by counting the number of moving vehicles at a predetermined location for a predetermined time (e.g., 1 hour), and as an example, the traffic volume can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. As an example, if the minimum traffic volume is 0, the maximum traffic volume is 3000 vehicles, and the actual traffic volume is 2000 vehicles, the traffic volume can be normalized to 0.667.

The average vehicle speed is the average of the moving speeds of each vehicle aggregated from the traffic volume measurements. For example, the average vehicle speed can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum average vehicle speed is 0, the maximum average vehicle speed is 120 km/h, and the actual average vehicle speed is 40 km/h, the average vehicle speed can be normalized to 0.333.

The accident history represents the number of accidents that occurred during a predetermined period (e.g., 1 year) within a predetermined section (e.g., 30 km radius from the measurement location) from the traffic volume measurement location. For example, the accident history can be normalized to 0 to 1 through min-max normalization within a predetermined minimum and maximum value range. For example, if the minimum accident history is 0, the maximum accident history is 20, and the actual accident history is 10, the accident history can be normalized to 0.5.

For example, when k is 10 and the normalized values of traffic volume, average vehicle speed, and accident history are 0.67, 0.33, and 0.5, respectively, the traffic information score is calculated as follows.

The generated traffic information score can be generated in real time according to the movement of vehicles at a specific location on each road, and as an indicator of the risk according to the traffic elements of each road, a high traffic information score can mean that the risk of the road according to the traffic information at that specific location is high. Through this, the road information analysis unit (510) can independently determine the road risk factors by utilizing both 'road information' and 'road image information'.

Meanwhile, the traffic information score can be normalized to 0 or 1 based on the minimum and maximum traffic information scores set in advance. Through this, the calculation of the road information score described below can be balanced with other indicators.

The road information analysis unit (510) can calculate a road information score based on the calculated structural information score, environmental information, and traffic information score as shown in the mathematical formula below. Here, w1 to w3 are predetermined weights and their sum can be 1.

The 'Road Information Score' is an indicator that represents an overall measure of road safety by combining structural, environmental and traffic data, and can become a single indicator representing risk factors for the road itself.

The road image information analysis unit (520) can obtain road image information by analyzing a received road image captured or received through a camera.

Here, the road image information is information obtained from the received road image, and may include potholes, cracks, lane markings, debris, water accumulation, skid marks, manhole covers, fallen objects, stopped vehicles, construction barriers, crosswalks, speed bumps, and signs.

When the road image information analysis unit (520) receives road image information, it can classify the received road image information into at least one category based on at least one attribute.

For example, the road image information analysis unit (520) can classify potholes, cracks, lane markings, debris, water accumulation, skid marks, and manhole covers as surface condition information, and classify fallen objects, stopped vehicles, construction barriers, crosswalks, speed bumps, and signs as obstacle information.

The road image information analysis unit (520) classifies road image information classified into surface condition information, obstacle information, and traffic information into each category and manages them, thereby enabling real-time monitoring of road image information and early identification and management of any deviations from the management pattern. In addition, the convenience of information management can be increased by managing the index of information classified into each category.

Meanwhile, the classification of road image information categories and the calculation of road image information scores can use artificial intelligence-based machine learning or deep learning models.

For example, artificial intelligence learning models including supervised learning, unsupervised learning, reinforcement learning, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), natural language processing (NLP), decision support systems (DSS), and predictive analytics can be used.

In addition, the road image information analysis unit (520) can monitor information classified into each category in real time and generate a notification (e.g., sound, vibration, light, etc.) for warning when a predetermined deviation range (e.g., 20%) is exceeded.

Next, the road image information analysis unit (520) can calculate a road image information score by mutually combining road image information classified into each category.

For example, the road image information analysis unit (520) can calculate a surface condition information score as shown in the mathematical formula below.

Here, the risk of potholes is calculated as follows based on the risk of potholes calculated through image analysis.

Here, the number of vehicles detecting potholes refers to the number of vehicles equipped with a road hazard control device based on image analysis that detected potholes within a predetermined section from a predetermined location, and the total number of vehicles traveling may refer to the total number of vehicles equipped with a road hazard control device based on image analysis that drove within a predetermined section from a predetermined location, including vehicles that failed to detect potholes.

Additionally, the average travel speed may mean the average travel speed of all vehicles traveling from a predetermined location to a predetermined section, and may be given a predetermined weight ( ) is a weight indicating the sensitivity to the risk of potholes, which can be predetermined as a value between 0 and 1, and is not limited thereto, and can be determined as 1 and applied if there is no separate setting.

For example, if the number of pothole detection vehicles is 30, the total number of vehicles passing through is 60, the average speed is 50 km/h, and the pre-determined weights are pre-determined ( ) is 1, the pothole is calculated as follows.

Meanwhile, as an example, a pothole can be normalized from 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum pothole is 0, the maximum pothole is 120, and the actual pothole is 25, the pothole can be normalized to 0.21. That is, a pothole, which means a pothole risk, indicates that the more vehicles detect a pothole among all passing vehicles, the deeper or larger the pothole is and that it has a significant risk enough to be detected through image analysis, and it can mean that a faster average travel speed increases the risk of an accident due to a pothole.

Additionally, potholes can be calculated as follows by reflecting the wheel size (in inches) of a vehicle equipped with a road hazard control device based on image analysis.

That is, the larger the wheel size of a vehicle equipped with a road hazard control device based on image analysis, the lower the risk of potholes.

Additionally, cracks can mean the total length of cracks in the road detected in images taken by a vehicle equipped with an image analysis-based road hazard control device, and can be calculated as follows by a vehicle driving within a predetermined section from a predetermined location.

The vehicle driving distance refers to the distance driven by a vehicle equipped with a road hazard control device based on image analysis, and the total length of cracks As a vehicle driving distance, the distance identified on the road driven by a vehicle equipped with a road hazard control device based on image analysis It can mean the total extension of the length.

For example, when the total crack extension is 1 km and the vehicle travel distance is 5 km, the crack can be calculated as 0.2, and when the total crack extension exceeds the vehicle travel distance, the crack can be determined as 1.

Lane marking can be calculated as follows as the ratio of distance where lanes are recognized on a road driven by a vehicle equipped with a road hazard control device based on image analysis.

For example, when the lane recognition distance is 4 km and the vehicle driving distance is 5 km, the lane marking can be calculated as 0.2.

The generated surface condition information score can be used to control road hazards by receiving image information on the surface condition of the road on which the vehicle is driving and judging the level of risk. A higher surface condition information score means a higher risk of the road surface as a result of image analysis.

The road image information analysis unit (520) can calculate obstacle information scores as shown in the mathematical formula below.

Here, the number of fallen objects is calculated as the number of fallen objects detected from a predetermined location during a predetermined section on a road on which a vehicle equipped with a road hazard control device based on image analysis drove. For example, if the number of fallen objects detected from a predetermined location during a predetermined section of 10 km is 10, the number of fallen objects can be normalized to 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum number of fallen objects is 0, the maximum number of fallen objects is 20, and the actual number of fallen objects is 10, the number of fallen objects can be normalized to (1-0.5)=0.5.

The number of stopped vehicles is calculated as the number of stopped vehicles detected from a predetermined location during a predetermined section on a road driven by a vehicle equipped with a road hazard control device based on image analysis. For example, if the number of stopped vehicles detected from a predetermined location during a predetermined section of 10 km is 10, the stopped vehicles can be normalized from 0 to 1 through min-max normalization within a range of predetermined minimum and maximum values. For example, if the minimum number of stopped vehicles is 0, the maximum number of stopped vehicles is 20, and the actual number of stopped vehicles is 10, the dropped object can be normalized to (1-0.5)=0.5.

Construction barriers can refer to the total length of construction barriers on the road detected in images taken by vehicles equipped with image analysis-based road hazard control devices, and can be calculated as follows by a vehicle driving within a predetermined section from a predetermined location.

The vehicle driving distance refers to the distance driven by a vehicle equipped with a road hazard control device based on video analysis, and the total length of the construction barrier refers to the distance driven by a vehicle equipped with a road hazard control device based on video analysis. As a vehicle driving distance, the distance identified on the road driven by a vehicle equipped with a road hazard control device based on image analysis It can mean the total extension of the length.

For example, when the total length of the construction barrier is 1 km and the vehicle travel distance is 5 km, the construction barrier can be calculated as 0.2.

The resulting obstacle information score provides a single measurement that can quantify the potential risk posed by obstacles on a road segment, and the results of image analysis can be used to assess road safety and normalize road risks across different obstacle factors.

The road image information analysis unit (520) can calculate the road image information score based on the calculated surface condition information score and obstacle information score as shown in the mathematical formula below. Here, w1 to w2 are predetermined weights and their sum can be 1.

The generated road image information score can be used as an indicator of the risk of the entire road by considering the risk factors of the road surface and the risk factors of obstacles on the road through image analysis.

The road hazard factor analysis unit (530) can calculate a road hazard factor score from a weighted sum of the calculated road information score and road image information score, and the road hazard factor score can be normalized to 0 or 1 according to a predetermined minimum and maximum road hazard factor score.

The road risk factor analysis unit (530) can adjust the number of communications with the external integrated control server according to the calculated road risk factor score.

For example, the road risk factor analysis unit (530) can calculate an adjusted communication frequency by adjusting the preset basic communication frequency.

Here, the basic number of communications is a value predetermined by the road risk factor analysis unit (530), and may mean the number of communications with an external integrated control server during a predetermined time (e.g., 1 hour).

At this time, communication may include communication for confirming the status of communication connection with an external integrated control server, vehicle status, video transmission, risk factor transmission, road information score and road video information score transmission, warnings, destination information, traffic information, and software updates.

For example, when the basic communication frequency is 10 times/hour and the road hazard factor score normalized from 0 to 1 is 0.5, the number of adjustment communications is calculated as follows.

That is, the number of adjustment communications may mean that the number of communications needs to increase depending on the road risk factor score, and depending on the road risk, it may communicate with an external integrated control server to report the road risk and receive road information statistics.

In addition, the road risk factor analysis unit (530) can calculate an adjusted communication bandwidth by adjusting a predetermined basic communication bandwidth.

Here, the basic communication bandwidth can be determined in advance as a communication bandwidth predetermined by the road risk factor analysis unit (530), for example, 2 Mbps. In this case, if the pre-calculated road risk factor score is 0.8, the adjusted communication bandwidth is calculated as follows.

That is, the adjusted communication bandwidth can communicate with the external integrated control server to reflect the road risk status, and the higher the road risk factor score, the wider the communication bandwidth can be used to increase the bandwidth so that low delay and high data processing can be handled.

Through this, the road risk factor analysis unit (530) can respond flexibly and efficiently even in road risk situations.

Fig. 6 is a hardware configuration diagram for a road hazard control device based on image analysis according to Fig. 1.

Referring to FIG. 6, the image analysis-based road hazard control device (600) may include at least one processor (610) and a memory (420) that stores instructions that instruct the at least one processor (610) to perform at least one step.

Here, at least one processor (610) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory (620) and the storage device (660) may be configured with at least one of a volatile storage medium and a nonvolatile storage medium. For example, the memory (620) may be configured with at least one of a read only memory (ROM) and a random access memory (RAM).

In addition, the image analysis-based road hazard control device (600) may include a transceiver device (630) that performs communication via a wireless network. In addition, the image analysis-based road hazard control device (600) may further include an input interface device (640), an output interface device (650), a storage device (660), etc. Each component included in the image analysis-based road hazard control device (600) may be connected by a bus (bus, 670) and communicate with each other.

The methods according to the present invention may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable medium may be those specially designed and configured for the present invention or may be those known and available to those skilled in the art of computer software.

Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions may include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, and the like. The above-described hardware devices may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

In addition, the above-described method or device may be implemented by combining all or part of its configuration or function, or may be implemented separately.

Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and field of the present invention as set forth in the claims below.

Claims (3)

As a road hazard control device based on image analysis,
at least one processor; and
A memory storing instructions that instruct at least one processor to perform at least one step,
At least one of the above steps,
Step of receiving road information from an external server;
A step of analyzing road images to obtain road image information;
A step of calculating a structural information score, an environmental information score and a traffic information score by mutually combining the above road information, and calculating a road information score from a weighted sum of the calculated structural information score, the environmental information score and the traffic information score;
A step of calculating a surface condition information score and an obstacle information score by mutually combining the above road image information, and calculating a road image information score from a weighted sum of the calculated surface condition information score and obstacle information score;
A step of calculating a road risk factor score from a weighted sum of the calculated road information score and the road image information score; and
A step of calculating the number of adjusted communications by multiplying the calculated road risk factor score by the number of basic communications determined in advance; including;
Video analysis-based road hazard control device.
In claim 1,
The step of calculating the above road information score is:
The structural information score is calculated from the sum of the reciprocal of the road width, the reciprocal of the road slope, and the reciprocal of the curvature radius.
The environmental information score is calculated from the sum of the values obtained by multiplying the reciprocal of the road surface temperature, the square of the reciprocal of the visibility, and the amount of precipitation and the wind speed.
The traffic information score is calculated by calculating the road information score from the sum of the sigmoid function values for the normalized values of traffic volume, average vehicle speed, and accident history.
Video analysis-based road hazard control device.
In claim 1,
The step of calculating the above road image information score is:
The surface condition information score is calculated from the weighted sum of potholes, cracks, and lane markings.
Characterized in that the obstacle information score is calculated from the sum of the sigmoid function values for the normalized values of falling objects, stopped vehicles, and construction barriers.
Video analysis-based road hazard control device.

Images