KR102787497B1 - Method for validating control signal of autonomous driving vehicle in teleoperated driving state and apparatus and system therefor - Google Patents

Abstract

The present invention relates to a method for verifying the validity of a control command of an autonomous vehicle in a remote-controlled driving state, and a device and system therefor. According to one aspect of the present disclosure, a method for verifying the validity of a control signal in a vehicle linked with a remote control center through a network may include a step of receiving the control signal while operating in a remote-controlled mode, a step of measuring a reception interval of the control signal, a step of performing control signal validity verification based on the measured reception interval, and a step of performing exception processing based on a result of the control signal validity verification.

Description

Method for validating control signal of autonomous driving vehicle in teleoperated driving state and apparatus and system therefor

The present invention relates to an autonomous vehicle, and more particularly, to a technology for verifying the validity of a control signal received from a remote control center in an autonomous vehicle in a remote control driving state.

Autonomous vehicles are vehicles that can drive themselves without the intervention of a driver or passenger. In addition, as communication technology advances, high-speed, large-capacity data transmission becomes possible, allowing for more diverse services to be provided through wireless communication systems.

Current autonomous vehicles are not yet technically capable of operating in environments such as heavy rain, heavy snow, thick fog, or unexpected situations. When Google received a license for a self-driving car in Nevada, the inspector pointed out problems with the vehicle’s ability to adapt to various weather conditions and environments such as unpaved roads.

To complement these problems of autonomous vehicles, research is actively being conducted on a remote control autonomous driving control system, that is, teleoperated driving (ToD), which can remotely monitor and operate autonomous vehicles at all times based on information about the autonomous vehicle's driving point, location information, and various sensing information collected by the autonomous vehicle from a remote location. As various transportation methods and services become popular and expanded, remote control of autonomous vehicles is expected to become a very important transportation element.

For safe remote control of autonomous vehicles, the reliability and validity of control commands must be guaranteed.

If the control command received from the remote control center is received with a delay by the vehicle being remotely controlled, vehicle control may be delayed, increasing the risk of an accident.

For example, suppose that a remote control center generates a control command for steering wheel control at regular time intervals (e.g., 10 ms). If a transmission delay or transmission cycle error of the control signal occurs due to a problem in the internal system of the remote control center, or a transmission delay of the steering wheel control signal occurs due to various problems that occur momentarily in the network, as illustrated in Fig. 1, the vehicle's rotation trajectory may unfold in a direction different from that intended by the remote driver.

Of course, the remote driver can monitor the remote driving screen to recognize the above-mentioned problem situation and adjust the steering wheel of the remote driving device, but this can not only increase the remote control fatigue of the remote driver, but also cause the vehicle to be controlled very unstably due to large and small control signal delays, which can reduce the ride comfort and increase the probability of various safety problems occurring.

Korean Patent Publication No. 10-2018-0052673 (May 18, 2018) relates to a method for estimating system delay for controlling an autonomous vehicle, and discloses a technology for measuring various delays within an autonomous driving system and generating control data based on an estimated overall system delay based on the measured delays.

Korean Patent Publication No. 10-2021-0010994 (January 26, 2021) relates to a vehicle platform that controls the vehicle in conjunction with an autonomous driving system according to commands from the autonomous driving system. In particular, the autonomous driving system transmits a first signal indicating whether it is in an autonomous mode or a manual mode to the vehicle platform, and when a second signal indicating the direction of travel of the vehicle is acquired, a method is disclosed in which a shift change is performed according to the first signal only when the first signal indicates an autonomous mode and the second signal is a stop signal.

Korean Patent Publication No. 10-2020-0055596 (May 21, 2020) discloses a technology in which a vehicle terminal device acquires multiple input images from multiple cameras and transmits them to a remote control device via a network, and the remote control device configures a packing image based on the received images.

Korean Patent Publication No. 10-2018-012625 (2018.11.27) discloses a technology in which a remote control device generates a path point and acceleration/deceleration commands for an unmanned vehicle to follow based on environmental information maps and image information generated from various sensors mounted on the unmanned vehicle.

The purpose of the present disclosure is to provide a method for verifying the validity of a control signal of an autonomous vehicle in a remote-controlled driving state and a device and system therefor.

Another purpose of the present disclosure is to provide a method for verifying the validity of a control signal of an autonomous vehicle in a remote-controlled driving state, and a device and system therefor, which can ensure the stability of the remote-controlled vehicle by verifying the validity of network delay as well as control signal delay due to an internal problem of a remote control center, and performing a warning alarm and switching to an autonomous driving mode according to the verification result.

Another object of the present disclosure is to provide a control signal validation device independently mounted inside a vehicle, which is designed to be capable of validating at least one of a control signal delay and a network delay.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

A method for verifying the validity of a control signal in a vehicle that is linked to a remote control center via a network according to one aspect of the present disclosure may include a step of receiving the control signal while operating in a remote control mode, a step of measuring a reception interval of the control signal, a step of performing the control signal validity verification based on the measured reception interval, and a step of performing exception processing based on a result of the control signal validity verification.

As an example, the step of performing the validation may include the step of identifying a vehicle device type corresponding to the received control signal, the step of setting a first threshold condition based on the identified vehicle device type, and the step of determining the validity of the control signal based on whether the measured reception interval satisfies the first threshold condition.

In an embodiment, the method further includes a step of establishing a 1:1 dedicated communication session with the remote control center based on entering the remote control mode, a step of establishing a second threshold condition for verifying network validity, a step of measuring a network delay time of a signal transmitted and received through the dedicated communication session, and a step of comparing the measured network delay time with the second threshold condition to determine network delay validity, and the exception handling can be performed further based on a result of the network delay validity determination.

In an embodiment, at least one of the first threshold condition and the second threshold condition may be dynamically set based on at least one of a vehicle driving state and an external environmental variable.

As an example, the vehicle driving state may include at least one of a current driving speed and a current road type.

As an example, the external environmental variables may include at least one of weather, season, and time zone.

As an example, the vehicle device type may include at least one of an accelerator, a braking device, a steering device, a lamp, multimedia, an air conditioning device, a wiper, a heating element, a door lock, a sunroof open/close device, and a window control device.

As an example, the first and second critical conditions may be maintained and managed in a database or file format in a storage provided in the vehicle.

As an example, the signal transmitted and received via the dedicated communication session may be a TCP-based ping signal.

As an example, the exception handling may include at least one of an action of outputting a warning alarm, an action of switching from the remote control mode to the autonomous driving mode, and an action of controlling emergency parking.

As an example, the method may further include a step of logging and storing the control signal reception interval measurement result and the network delay measurement result.

In an embodiment, the control signal is received periodically in the form of a stream, and the reception interval of the control signal can be measured based on a difference value in reception times between two control signals received consecutively corresponding to the same vehicle device type.

In another aspect of the present disclosure, a nonvolatile computer-readable storage medium storing at least one computer program including instructions that, when executed by at least one processor, cause the at least one processor to perform validation operations on a control signal by interacting with a remote control center through a network, wherein the operations may include the steps of receiving the control signal while operating in a remote control mode, measuring a reception interval of the control signal, performing the control signal validation based on the measured reception interval, and performing exception handling based on a result of the control signal validation.

According to another aspect of the present disclosure, a vehicle interoperating with a remote control center through a network includes a remote driving controller for controlling remote driving, an autonomous driving controller for controlling autonomous driving, a transmitter and receiver for receiving a control signal while operating in a remote control mode, and a validation device for measuring a reception interval of the control signal and performing validation of the control signal based on the measured reception interval, wherein a validation result of the control signal is transmitted to at least one of the remote driving controller and the autonomous driving controller so that exception processing can be performed.

As an example, the validation device may identify a vehicle device type corresponding to the received control signal, set a first threshold condition based on the identified vehicle device type, and determine the validity of the control signal based on whether the measured reception interval satisfies the first threshold condition.

In an embodiment, based on the fact that the validation device has entered the remote control mode, a 1:1 dedicated communication session is established with the remote control center, a second threshold condition for network validation is established, a network delay time of a signal transmitted and received through the dedicated communication session is measured, and the network delay validity is determined by comparing the measured network delay time with the second threshold condition, wherein the exception handling may be performed further based on the network delay validity determination result.

In an embodiment, at least one of the first threshold condition and the second threshold condition may be dynamically set based on at least one of a vehicle driving state and an external environmental variable.

As an example, the vehicle driving state may include at least one of a current driving speed and a current road type.

As an example, the external environmental variables may include at least one of weather, season, and time zone.

As an example, the vehicle device type may include at least one of an accelerator, a braking device, a steering device, a lamp, multimedia, an air conditioning device, a wiper, a heating element, a door lock, a sunroof open/close device, and a window control device.

As an example, the first and second critical conditions may be maintained and managed in a database or file format in a storage provided in the vehicle.

As an example, the signal transmitted and received via the dedicated communication session may be a TCP-based ping signal.

As an example, the exception handling may include at least one of a means for outputting a warning alarm to the vehicle, a means for switching from the remote control mode to the autonomous driving mode, and a means for controlling emergency parking.

As an example, the vehicle may further include means for logging and storing the results of measuring the interval between receiving the control signal and the results of measuring the network delay.

In an embodiment, the control signal is received periodically in the form of a stream, and the reception interval of the control signal can be measured based on a difference value in reception times between two control signals received consecutively corresponding to the same vehicle device type.

According to another aspect of the present disclosure, a device mounted in a vehicle and interconnected with a remote control center via a network may include a validity judgment threshold database that maintains information on threshold conditions necessary for determining the validity of a control signal, a control signal interval measuring unit that measures a reception interval of the control signal when a control signal is received while operating in a remote control mode, a network signal delay measuring unit that establishes a 1:1 dedicated communication session with the remote control center when entering the remote control mode, transmits and receives a signal for network delay verification through the dedicated communication session, and measures network delay time, and a validity judgment unit that performs validity verification on the measured reception interval and the measured delay time by referencing the validity judgment threshold database, and controls that predetermined exception processing is performed based on a result of the validity verification.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

Various embodiments according to the present disclosure have the advantage of providing a method for verifying the validity of a control signal of an autonomous vehicle in a remote-controlled driving state and a device and system therefor.

In addition, various embodiments according to the present disclosure have the advantage of providing a method for verifying the validity of a control signal of an autonomous vehicle in a remote-controlled driving state, and a device and system therefor, which can secure the stability of the remote-controlled vehicle by verifying the validity of a delay due to a network problem as well as a control signal delay due to an internal problem of a remote control center, and performing a warning alarm and a transition to an autonomous driving mode according to the verification result.

In addition, various embodiments according to the present disclosure have the advantage of providing a control signal validation device independently mounted inside a vehicle that is designed to verify at least one of a delay in a control signal received from a remote control center to a remote driving vehicle and a signal delay on a network.

The effects that can be obtained in various embodiments are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below.

The drawings appended hereto are included to provide an understanding of the present invention and to illustrate various embodiments of the present invention and, together with the description of the specification, serve to explain the principles of the present invention.
Figure 1 is a drawing to explain a problem caused by control signal delay during remote driving.
FIG. 2 is a drawing illustrating a remote driving system of one embodiment of the present disclosure.
FIG. 3 is a drawing for explaining a detailed configuration of a remote driving system according to an embodiment of the present disclosure.
FIG. 4 is a drawing for explaining the general operation of a remote-controlled vehicle according to one embodiment of the present disclosure.
FIG. 5 is a drawing for explaining the detailed structure of a remote driving system according to an embodiment of the present disclosure.
FIG. 6 is a detailed configuration diagram of a validation device according to an embodiment of the present disclosure.
FIG. 7 is a diagram for explaining the structure of a validity judgment threshold database according to an embodiment of the present disclosure.
FIG. 8 is a drawing for explaining a detailed configuration of a control signal interval measuring unit according to an embodiment of the present disclosure.
FIG. 9 is a drawing for explaining a detailed configuration of a network signal delay measurement unit according to an embodiment of the present disclosure.
Figure 10 shows the experimental results of measuring network delay time according to the present disclosure.
FIG. 11 is a drawing for explaining the detailed configuration of a validity judgment unit according to an embodiment of the present disclosure.
FIG. 12 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to an embodiment of the present disclosure.
FIG. 13 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to another embodiment of the present disclosure.
FIG. 14 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to another embodiment of the present disclosure.

Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing symbols, identical or similar components will be given the same reference numerals and redundant descriptions thereof will be omitted. The suffixes "부" and "부" used for components in the following description are given or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

Terms including ordinal numbers such as first, second, etc. may be used to describe various components, but the components are not limited by these terms. These terms are used only to distinguish one component from another. When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, it should be understood that terms such as “comprises” or “has” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Hereinafter, with reference to FIGS. 1 to 14, a method for verifying the validity of a control command of an autonomous vehicle in a remote driving state according to the present disclosure and a device and system therefor will be described in detail.

Figure 1 is a drawing to explain a problem caused by control signal delay during remote driving.

Drawing symbol (a) of Fig. 1 shows a normal moving trajectory of a vehicle when a steering wheel control signal transmitted from a remote control center is received by the remote driving vehicle without delay at a certain period - for example, 10 ms.

Drawing symbol (b) of Fig. 1 shows an abnormal movement trajectory of a vehicle when a steering wheel control signal transmitted from a remote control center is received with a delay by the remote driving vehicle.

As shown in the drawing number (b), if the second steering wheel control signal is received with a delay for an unknown reason after the first steering wheel control signal is received with a normal cycle, the vehicle begins to deviate from the normal driving trajectory, and if the third steering wheel control signal is received by the vehicle with a delay again, the vehicle may deviate from the set road. In this case, the remote driver must change the steering wheel angle very greatly and frequently in order to maintain the normal driving trajectory. Accordingly, the vehicle drives in a zigzag manner, which not only reduces the riding comfort of the passengers but also increases the control fatigue of the remote driver. In addition, the unstable control signal delay phenomenon can drastically increase the risk of an accident of the vehicle.

The reason for the delay in the steering wheel control signal may be caused by a processing delay inside the remote control center, and in this case, the problem can be solved by resolving the delay problem inside the remote control center.

However, the reason for the delay in the steering wheel control signal can be caused not only by problems within the remote control center but also by network problems. Since the control signal delay due to network problems is not a problem of the remote driving vehicle and/or the remote control center itself, the remote driving vehicle and/or the remote control center cannot accurately recognize the cause of the control signal delay, which may increase the risk of an accident.

The channel environment of the network can change rapidly depending on the movement of the vehicle, regional characteristics, and current network operation status, and for stable remote control, it is necessary for the autonomous vehicle and/or remote control center to accurately anticipate and recognize the network delay problem and take appropriate measures accordingly.

FIG. 2 is a drawing illustrating a remote driving system of one embodiment of the present disclosure.

A tele-operated driving (ToD) system (100) is a technology that directly or indirectly monitors and controls a vehicle equipped with an autonomous driving function, i.e., an autonomous vehicle, through a wired/wireless network, and a remote server, through remote control (tele-operated) from a remote center (or server) when a problem occurs during operation of the autonomous vehicle. It is being researched and developed in various fields recently.

Remote driving technology is a technology in which a remote driver operates a remote driving device equipped in a remote center (120) to remotely operate an autonomous vehicle (110). The remote driving system (100) is largely composed of three components.

Referring to FIG. 2, the remote driving system (100) may be composed of a tele-operated vehicle (ToV, 110), which is an autonomous vehicle capable of remote control, a network (network, 130), and a tele-operated center (ToC, 120) that remotely controls the vehicle. At this time, the remote-controlled vehicle (110) is capable of autonomous driving and is capable of encoding an environment model (E/M). In addition, the network (130) may use a 5G communication network, but is not limited thereto, and a 4G communication network, a 6G communication network, or another mobile communication network may be used according to the design of a person skilled in the art. In addition, the remote control center (120) is capable of decoding an environment model (E/M) and may be capable of remote monitoring and remotely operating the vehicle through a display.

The remote control vehicle (110) is a remote control target vehicle, and must be equipped with an autonomous driving function and may be equipped with various safety sensors required for autonomous driving. Here, the safety sensors may include, but are not limited to, cameras, radar, lidar, and sonar (Sound Navigation And Ranging, SONAR) sensors for implementing an ADAS (Advanced Driver Assistance System).

A camera is a vehicle sensor equivalent to the human eye, and can be used to visually recognize surrounding objects through the lens and to recognize the driving situation through the recognized surrounding objects.

The camera can process image frames, such as still images or moving images, obtained by the image sensor. At this time, the processed image frames can be displayed on the display or stored in the memory. In addition, multiple cameras installed in the autonomous vehicle can be arranged to form a matrix structure, and multiple image information having various angles or focal points can be input through the cameras forming such a matrix structure. In addition, multiple cameras can be arranged in a stereo structure to obtain left and right images for implementing a three-dimensional image.

In one embodiment of the present invention, the camera can sense driver view data of an autonomous vehicle. Here, the driver view data can include an image outside the vehicle.

Radar emits electromagnetic waves and can extract information such as distance, speed, and direction from surrounding objects based on the reflected signals. Depending on the frequency used, radar can detect short, medium, and long distances, and can be used for emergency automatic braking systems, smart cruise control, etc. If three radar sensors are installed side by side on a self-driving car, it can secure a 180-degree field of view ahead. Radar is a sensor that is widely used in vehicles currently in operation because it is not affected by weather, etc., and can detect the presence of obstacles at a distance.

Lidar can recognize the perspective, shape, distance, and speed of objects by exchanging lasers (light) with them to form a three-dimensional map. Lidar mainly uses a short wavelength of 905 nanometers (nm), so it has much more precise spatial resolution than radar, and has the advantage of being less affected by light in environments with insufficient light because it has its own light source. Therefore, Lidar plays an important role in increasing the reliability of autonomous driving systems.

Sonar is a sensor that detects objects in front or behind and measures the distance to the objects by measuring the time it takes for sound waves to be transmitted, bounce off an object, and be received, rather than electromagnetic waves. Sonar is mainly used to detect objects in the blind spot behind a vehicle when it is backing up and to notify the driver. Since sound waves propagate at a much slower speed than electromagnetic waves, it has the advantage of being able to identify even small objects with high resolution.

A remote control vehicle (110) can provide an optimal driving algorithm for autonomous driving by applying sensor fusion technology and AI technology that combine and fuse sensing data collected from each sensor.

The remote control vehicle (110) can transmit the collected sensing data to the remote control center (120) via a network (130).

The remote control center (120) can generate control data based on the collected sensing data and transmit the generated control data to the remote control vehicle (110) through the network (130).

Here, the environmental model corresponds to modeling of the surrounding environment data using vehicle sensors (speed, position, direction, vehicle condition) that provide sensing information for identifying the vehicle's speed/position/direction/vehicle condition, etc., and autonomous driving sensors that recognize surrounding objects and estimate the movement trajectories of recognized objects to control the vehicle's driving - for example, lidar, radar, sonar, V2X communication, cameras, etc.

In particular, in order to implement a remote driving system (100), a network adaptation technology for overcoming a communication environment is absolutely necessary, and includes uplink and downlink technologies and autonomous driving technologies. Among these, the uplink technology is related to transmitting sensing data from images and sensors, and the downlink technology may be related to generating and transmitting control data for controlling a remotely controlled vehicle (110) from a remote control center (120).

Below, uplink transmission is described.

A remote control vehicle (ToV, 110) can encode at least two environmental models and transmit them to a remote control center (ToC, 120). At this time, the remote control vehicle (110) can encode an environmental model including sensing data through an encoder and transmit it to the remote control center (120) through a network (130, for example, 5G). Meanwhile, the remote control center (120) can decode the received environmental model through a decoder and output it through an equipped display.

At this time, the two environmental models may include driver vision data and vehicle sensing data. At this time, the driver vision data may compress and transmit vehicle exterior images (4 channels or 2 channels), and the vehicle sensor data may include sensing information about vehicle position and sensing information about vehicle driving status. Sensing information about vehicle driving status may include, but is not limited to, information about driving speed, braking control information, acceleration (accelerator) control information, steering control information, and impact detection information.

To this end, low-latency video communication technology, fast and stable network technology, and low-latency display technology are required. Through these, it is possible to minimize video and network delay and latency to achieve accurate and fast data communication.

Below, downlink transmission is described.

The remote control center (ToC, 120) can detect the status of the remote control vehicle (110) to generate direct/indirect control signals (and/or control commands) and transmit the generated control signals to the remote control vehicle (110). Here, the direct control signals may include control data for controlling the vehicle driving device. Accordingly, the remote control center (ToC, 120) can generate and transmit control data for the vehicle driving device. In addition, the indirect control signals may include driver guide data. Accordingly, the remote control center (120) can generate driver guide data and transmit it to the remote control vehicle (110).

To this end, vehicle status and control type inference technology through understanding the transmitted environment model is required, and accurate vehicle control type definition through understanding the environment model becomes important.

A remote control center (120) according to an embodiment may calculate a remote driving trajectory path (or waypoint and/or route and/or track) for a specific road section - for example, a road section where forward autonomous driving is not possible - based on an environmental model collected from a remotely controlled vehicle (110), and may register and store the calculated remote driving trajectory path in an external cloud server, a local edge server, or a private server. At this time, the registered remote driving trajectory path may be shared with other autonomous vehicles passing through the corresponding road section, and the other autonomous vehicles may perform vehicle control on the corresponding road section based on the obtained remote driving trajectory path.

For example, information about a remote driving trajectory path (or simply remote driving information) may be composed of at least one of a waypoint, a route, and a track. Here, a waypoint may mean a waypoint, a point of interest, or a specific named entity on a map. A route may mean a sequential list of waypoints representing a series of turn points leading to a destination. A track may mean a sequential list of trackpoints describing a path.

FIG. 3 is a drawing for explaining a detailed configuration of a remote driving system according to an embodiment of the present disclosure. Hereinafter, any part that overlaps with the description given above in FIG. 2 will be omitted.

Referring to FIG. 3, the remote driving system (200) may be configured to include a remote control vehicle (210), a data encoding unit (211), a first network adaptation unit (212), a remote control center (220), a data decoding unit (221), a second network adaptation unit (222), and a network (230). However, here, the meaning that the remote driving system (200) includes all of the above-described configurations does not mean that they are physically included, but rather that they are interconnected and operable within the remote driving system (200).

The remote control vehicle (210) can compress and/or encode sensed data through the data encoding unit (211) and transmit it to the remote control center (220). At this time, the first network adaptation unit (212) can monitor the status of the network (230) and adjust various system parameters for smooth communication.

In addition, the remote control center (220) can receive, decrypt, and/or decompress sensing data transmitted by the remote control vehicle (210) through the data decryption unit (221). For example, the sensing data can include data collected from a vehicle sensor, image data captured by a camera, etc.

At this time, the first network adaptation unit (212) can perform the logic of the remote control vehicle (210), and the second network adaptation unit (222) can perform the logic of the remote control center (220).

The first network adaptation unit (212) and/or the second network adaptation unit (222) can adaptively adjust the compression ratio of the video compression system by predicting the current network status - for example, the allocated bandwidth status, the available power status, the channel occupancy status, etc. To this end, the first network adaptation unit (212) and/or the second network adaptation unit (222) can adaptively adjust the required network bandwidth and the payload of all communicable networks to efficiently transmit a large amount of compressed video information to the remote control center (220) via a wireless network by performing Channel Bonding, Channel Estimation, Channel Optimization, etc.

FIG. 4 is a drawing for explaining the general operation of a remote-controlled vehicle according to one embodiment of the present disclosure.

Referring to FIG. 4, the remote control vehicle can be largely configured to include an information providing subject (301), a processing and judgment subject (302), and an action subject (303).

The information provider (301) can provide high-precision map information and various sensing information to the processing and judgment subject (302).

As illustrated in FIG. 4, the information provider (301) may include a high-precision map storage, a safety sensor, and a vehicle sensor.

High-definition maps (or HD maps) contain detailed information about the surface of a road or intersection, including lanes, intersections, construction zones, and road signs. In addition to simply determining the location of autonomous vehicles, HD maps can provide a variety of information to determine the route the vehicle needs to take.

For example, safety sensors may include cameras, sonar sensors, lidar, radar, etc., and vehicle sensors may include wheel sensors, inertial measurement units (IMUs), global navigation satellite systems (GNSS), etc.

GNSS and IMU can measure the position of the vehicle and provide measurement values for inertial information and geographical position to the processing and judgment subject (302) at a fast cycle of 200 Hz or more. A Kalman filter can be used to effectively combine the advantages and disadvantages of the slow cycle and high accuracy of GPS and the fast cycle and large accumulated error of IMU.

Lidar can be used for map mapping, localization, obstacle avoidance, etc. It can measure the ToF (Time of Flight) of laser light to measure distance and create a monochrome 3D map. Because Lidar has high accuracy, it can be mainly used for creating HD maps, localizing (estimating) the location of a moving vehicle, and detecting obstacles ahead.

Cameras can be used for object recognition and tracking tasks, such as detecting lanes, traffic lights, and pedestrians. For example, more than eight 1080p cameras can be used to enhance safety. Based on the camera sensing information, the processing and judgment subject (302) can detect and recognize objects in the front, rear, left/right sides, and track them.

Radar and sonar can be used as a last resort for obstacle avoidance. The sensing information from radar and sonar can provide distance and speed information to the nearest object along the vehicle's path.

The processing and judgment subject (302) may correspond to an autonomous driving controller.

The autonomous driving controller can be configured to include a high-precision positioning unit, a path generation unit, a V2X (Vehicle to Everything) communication unit, an autonomous driving judgment unit, a sensor fusion unit, a control command generation unit, and a remote driving connection unit.

A high-precision positioning unit can measure and/or estimate the position and attitude of a vehicle based on sensing information.

The path generation unit can generate a driving path of the vehicle based on sensing information.

The V2X communication unit can provide V2X communication functions. V2X communication refers to a communication technology that exchanges information with other vehicles, pedestrians, objects with built-in infrastructure, etc. through wired/wireless communication. V2X can be divided into four types: V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2N (vehicle-to-network), and V2P (vehicle-to-pedestrian). V2X communication can be provided through the PC5 interface and/or the Uu interface.

The autonomous driving judgment unit can control to enter autonomous driving mode if autonomous driving is possible according to the driver's autonomous driving request. In addition, if the autonomous driving judgment unit determines that autonomous driving is no longer possible during autonomous driving, it can control to switch to manual control mode. If the autonomous driving judgment unit determines that autonomous driving is possible again during operation in manual control mode, it can also control to switch to autonomous driving mode.

The sensor fusion unit can display information about the immediate surroundings of the vehicle on the HD-MAP by integrating the strengths and characteristics of the sensing information collected from each sensor.

Through sensor fusion, the high-precision positioning unit can perform lane-level high-precision positioning, and the path generation unit can generate a close-range path for the vehicle.

The control command generation unit can obtain short-range situation information through V2X communication, and comprehensively consider the high-precision positioning results and path generation results described above, and the short-range situation information obtained through V2X communication to recognize objects and track the location of the objects, and generate a control command for the action subject (303) based on this.

The remote driving linkage (or ToD linkage) can perform the function of transitioning from autonomous driving to remote driving, which is recently becoming legally required.

The remote driving linkage can switch the autonomous driving mode to the remote driving mode when autonomous driving is not possible on the front road section during autonomous driving, or when a request for control transfer is received from the remote control center, or when remote driving is requested from the driver.

The operating subject (303) may include an engine ECU (Electronic Control Unit), a braking ECU, a steering ECU, a shifting ECU, etc. The operating subject (303) may operate according to a control command received from a processing and judgment subject (302).

FIG. 5 is a drawing for explaining the detailed structure of a remote driving system according to an embodiment of the present disclosure.

Referring to FIG. 5, the remote driving system (400) can be largely configured to include a remote control vehicle (ToV, 410), a remote control center (ToC, 420), and a network (430).

A remote control vehicle (410) may be configured through at least one of a vehicle sensor (411), an autonomous driving controller (412), a vehicle ECU (413), a ToD camera (414), an image compressor (415), a remote driving controller (416), a network status predictor (417), a transceiver (418), a validation device (419), and an alarm device (431), or a combination thereof.

Although not shown in the above drawing 5, the remote control vehicle (410) may be configured to further include a map storage (not shown). The map storage may be used to maintain a high-precision map required for autonomous driving and to provide information about the high-precision map to the autonomous driving controller (412).

The vehicle sensor (411) can collect various sensing information received from safety sensors for ADAS and various sensing information from other sensors equipped in the vehicle and/or the vehicle ECU (413) and provide them to the autonomous driving controller (412).

For example, information collected by the vehicle sensor (411) may include information that can be easily obtained from the vehicle's OBD (On-Board Diagnostic) device, such as four-wheel wheel ticks, steering angle, speed, acceleration, vehicle attitude control, and tire pressure.

The vehicle ECU (413) may include various ECUs that operate according to control commands of the autonomous driving controller (412).

Specific descriptions of the vehicle sensor (411) and the vehicle ECU (413) are replaced with descriptions of the drawings described above.

The autonomous driving controller (412) according to the embodiment may request the remote driving controller (416) to switch to the remote driving mode when autonomous driving can no longer be maintained while driving in the autonomous driving mode or when a switch to the remote driving mode is requested at the request of the driver or a remote location.

For example, the autonomous driving controller (412) may determine that it is no longer difficult to maintain autonomous driving when it detects that there is no high-precision map information for the road section ahead, that it is impossible to identify an obstacle ahead, or that an external impact exceeding a standard value is detected.

The remote driving controller (416) can drive the ToD camera (414) when switched from autonomous driving mode to remote driving mode.

The image captured by the ToD camera (414) can be compressed by the image compressor (415) and then transmitted to the remote control center (420) by the transmitter/receiver (418). For example, the ToD camera (414) can capture four images of the front/rear/left/right of the remote control vehicle (410), and the vehicle image information transmitted to the remote control center (420) can include at least one of the four captured images.

If the remote driving controller (416) determines that the remote driving mode can no longer be maintained during remote driving, it can transmit a predetermined control signal to the autonomous driving controller (412) to request a transition to autonomous driving mode.

For example, the remote driving controller (416) may decide to switch from remote driving mode to autonomous driving mode if it determines that a control signal received from the remote control center (420) is invalid.

The network status predictor (417) can monitor the current network status and select a channel and/or session suitable for communication with the remote control center (420).

The network status predictor (412) can determine the compression ratio of the image captured by the ToD camera (414) based on the available transmission rate of the selected channel and/or session, and provide information about the determined compression ratio to the image compressor (415).

Image data compressed by an image compressor (415) and vehicle sensing data collected from a vehicle sensor (411) can be encoded and modulated through a transceiver (418) and transmitted through a wireless channel selected by a network status predictor (417).

For example, compressed image data and vehicle sensing data can be transmitted in time division multiplexing or frequency division multiplexing.

The remote driving controller (416) according to the embodiment may determine the compression ratio of the image compressor (415) based on the channel information selected by the network status predictor (417), and the image compressor (415) may perform image compression according to the determined compression ratio. For example, the better the channel status, the higher the image compression ratio may be determined, and the worse the channel status, the lower the image compression ratio may be determined.

The remote driving controller (416) can receive a vehicle control command from the remote control center (420) through the transmitter/receiver (418). Here, the vehicle control command can be received in the form of a stream of a certain period. For example, the control signal transmission period can be set differently depending on the type of the vehicle controller to be controlled. For example, a steering wheel-related control signal can be received in a first period, and an accelerator and brake-related control signal can be received in a second period different from the first period. Here, the control signal reception period and the allowable threshold range (or threshold value) of each vehicle controller can be defined in advance and then stored in a database and maintained in a storage or memory (not shown) of the remote control vehicle (410).

The remote driving controller (416) can transmit the received vehicle control command to the autonomous driving controller (412). The autonomous driving controller (412) can control the corresponding vehicle ECU (413) according to the received vehicle control command.

The remote control center (420) may be configured to include at least one or a combination of a remote control center controller (ToC controller, 421), a transmitter/receiver (422), an image decoder (423), a monitoring device (424), a remote driving device (425), and a network delay verification signal response device (426).

A remote driver can perform remote driving using a remote driving device (425) while monitoring an image displayed on a display screen. Here, the remote driving device (425) may be equipped with a means for controlling various vehicle functions such as an infotainment system, various lamps, and wipers, as well as basic driving control means such as a steering wheel, an accelerator pedal, a brake pedal, and a gear device.

A network delay verification signal reply device (426) according to an embodiment may, when a network delay verification signal is received from a remote control vehicle (410), measure the time at which the signal is received, generate a network delay verification reply signal including at least one of measured reception time information and reply signal transmission time information, and transmit the signal to the remote control vehicle (410). As an example, the network delay verification signal may be a ping signal, but is not limited thereto, and a separate signal may be defined and used.

The transceiver (422) can demodulate and decode a signal received through the network (430) and provide it to the remote control center controller (421) and the network delay verification signal response device (426).

The remote control center controller (421) can receive image information and vehicle sensor information from a remote control vehicle (410) through a transmitter/receiver (422). Here, the image information may be compressed image information.

The remote control center controller (421) transmits image information to the image decoder (423) to decompress it, transmits the decompressed image information to the monitoring device (424), and the monitoring device (424) can display the image information on a screen provided therewith.

A remote driver can operate the remote driving device (425) while viewing the monitoring screen.

A vehicle control command-or control signal or control command-generated by operation of a remote driving device (425) can be transmitted to a remotely controlled vehicle (410) via a remote control center controller (421) and/or a transmitter/receiver (422).

As described above, the remote control center (420) according to the present disclosure can be implemented so that the remote control vehicle (410) can measure network delay by having a network delay verification signal reply device (426).

When the remote control vehicle (410) enters the remote driving mode, it establishes a 1:1 dedicated communication session with the remote control center (420), and transmits a network delay verification signal at regular intervals through the established dedicated communication session to measure the network delay. Here, the dedicated communication session for measuring the network delay can be maintained until the remote driving mode is terminated.

The remote control vehicle (410) according to the embodiment may generate a predetermined alarm signal and transmit it to the remote control center (420) when a network delay is detected. In this case, a network delay warning alarm may be displayed on one side of the screen of the monitoring device (424) of the remote control center (420). The remote driver may recognize that a network delay is occurring through the warning alarm, and may perform remote control more carefully accordingly.

A remote driver according to an embodiment may also input a predetermined control command to request the remotely controlled vehicle (410) to switch to autonomous driving mode when remote control is no longer possible due to network delay.

FIG. 6 is a detailed configuration diagram of a validation device according to an embodiment of the present disclosure.

Referring to FIG. 6, the validation device (419) may be configured to include at least one of a validation judgment threshold database (510), a control signal interval measurement unit (520), a network signal delay measurement unit (530), a validation judgment unit (540), and an alarm unit (550).

The validity judgment threshold database (510) may maintain a predefined threshold range (or threshold value) required for judgment of the validity of a control signal. Here, the threshold range (or threshold value) may be defined based on the type of the control signal (or the type of the controlled target device) and environmental variables, etc. For example, the environmental variables may include weather, season, time zone, etc., but this is only one embodiment, and may be defined further based on traffic volume, road type, speed limit, etc. For example, the controlled target device may include, but is not limited to, a steering wheel, a brake, an accelerator, a lamp, an ignition, a door lock, a mirror switch, a wiper, a heater, an air conditioner, a panoramic sunroof, multimedia, an electronic convenience device, etc.

The control signal interval measuring unit (520) can measure the reception interval of two consecutive signals received in the form of a stream for each control signal type.

The network signal delay measurement unit (530) can measure network delay time based on the time at which a network delay verification signal is generated and transmitted to the remote control center (420) through a dedicated communication session established when entering the remote driving mode, and the time at which a network delay verification response signal corresponding to the network delay verification signal is received from the remote control center (420).

The validity judgment unit (540) can compare the reception interval of the corresponding control signal received from the control signal interval measurement unit (520) with a threshold range (or threshold) corresponding to the corresponding control signal to determine whether the corresponding control signal is valid.

The validity judgment unit (540) can compare the network delay time received from the network signal delay measurement unit (530) with a predefined network delay threshold range (or threshold) to determine whether the current network delay state is normal.

The validity judgment unit (540) can determine whether remote driving mode can be maintained based on the validity judgment result for the control signal and the network delay status judgment result.

If the validity judgment unit (540) determines that it is no longer possible to maintain the remote driving mode, it can generate a predetermined control signal requesting a transition to the autonomous driving mode and transmit it to the remote driving controller (416) or the autonomous driving controller (412).

If the validity judgment unit (540) according to the embodiment determines that it is no longer possible to maintain the remote driving mode, it may transmit a predetermined control signal to the alarm device (431) to request generation of a warning alarm.

The validity judgment unit (540) according to the embodiment may control the vehicle to perform a specific action - or exception processing - if it is determined that maintaining the remote driving mode is no longer possible. As an example, the specific action may include, but is not limited to, measures such as emergency parking guidance and turning on the emergency lamp.

The alarm device (431) can generate a predetermined warning alarm message according to a control signal received from the validity judgment unit (540) and transmit it to the remote control center (420).

The validity judgment unit (540) can adaptively control switching to autonomous driving mode and generating warning alarms according to the control signal reception interval and network delay time.

For example, if the control signal reception interval and the network delay time satisfy a specific first condition, the validity judgment unit (540) can control only the generation of a warning alarm, and if a specific second condition is satisfied, the generation of a warning alarm and the transition to autonomous driving mode can be controlled.

FIG. 7 is a diagram for explaining the structure of a validity judgment threshold database according to an embodiment.

Referring to Figure 7, the validity judgment threshold can be largely composed of a network delay tolerance threshold and a tolerance threshold range (or threshold) for each controlled target device.

For example, the controlled device may include, but is not limited to, a steering wheel, an accelerator, and a brake, and may include any device of the vehicle that can be controlled by a remote driver through a remote driving device (425).

In other embodiments, different validity judgment thresholds may be recorded and maintained in a given storage in a specific file format.

The validity judgment threshold range (or threshold) according to the embodiment may be defined multiple times for each controlled target device. For example, a maximum reception interval (Max Interval) may be defined for a steering wheel (handle). Here, the maximum reception interval may be defined differently depending on the current driving speed of the vehicle. For example, it may be defined as “80 km/h or more: 10 ms”, “50 km/h or more: 15 ms”, “30 km/h or more: 20 ms”. As another example, the maximum reception interval may be defined differently depending on the class or type of the road on which the vehicle is currently traveling. For example, it may be defined as “Class 1 road (expressway): 10 ms”, “Class 2 road (automobile-only road): 15 ms”, “Class 3 road (general road): 20 ms”, and “Class 4 road (alley): 25 ms”.

The network delay tolerance threshold according to the embodiment may be defined as one threshold, but this is only one embodiment, and the network delay tolerance threshold according to another embodiment may be defined and used as multiple thresholds according to the current driving speed of the vehicle, weather, time zone, etc. In this case, information about the weather may be updated by collecting information from an external weather provision server.

FIG. 8 is a drawing for explaining a detailed configuration of a control signal interval measuring unit according to an embodiment of the present disclosure.

Referring to FIG. 8, the control signal interval measuring unit (520) may be configured to include a reception interval measuring module (710), a control signal distribution module (720), and a reception time measuring module (730).

The reception interval measurement module (710) may be equipped with a module that calculates the reception interval of a control signal for each control signal type.

The reception time measurement module (730) can measure the reception time of a control signal. The reception time measurement module (730) can provide a control signal including measured reception time information to the control signal distribution module (730).

The control signal distribution module (740) can identify the type of the received control signal and determine a control signal reception interval calculation module to transmit the corresponding control signal based on the identified type. The control signal distribution module (740) can transmit a control signal including reception time information to the determined control signal reception interval calculation module.

Each control signal reception interval calculation module provided in the reception interval measurement module (710) can measure the reception interval of the corresponding control signal based on the reception time difference value between two control signals received continuously.

FIG. 9 is a drawing for explaining a detailed configuration of a network signal delay measurement unit according to an embodiment of the present disclosure.

Referring to FIG. 9, the network signal delay measurement unit (530) may be configured to include a session setup and management module (910) and a delay time measurement module (920).

The session setup and management module (910) can set up a 1:1 dedicated communication session with the remote control center (420) when entering the remote driving mode, and maintain the set dedicated communication session until the remote driving mode is released under an agreement between the remote control vehicle (410) and the remote control center (420). For example, the dedicated communication session may use a connection-based TCP communication session to support reliable communication, but this is only one embodiment, and a more reliable communication session may be used. The dedicated communication session may be used only for the purpose of measuring a network signal delay between the remote control vehicle (410) and the remote control center (420).

The delay time measurement module (920) can generate a network delay verification signal of a very small size and transmit the network delay verification signal to the remote control center (420) through a preset dedicated communication session. The delay time measurement module (920) can receive a response signal corresponding to the network delay verification signal - for example, a network delay verification reply signal - from the remote control center (420) through the dedicated communication session.

The delay time measurement module (920) according to the embodiment can measure the network delay time based on the difference value between the time of transmitting the response signal from the remote control center (420) and the time of receiving the corresponding response signal from the remote control vehicle (410). In this case, the response signal can include information about the time of transmitting the response signal from the remote control center (420).

In another embodiment, the delay time measurement module (920) can record information about the transmission time (a1) of the network delay verification signal in the internal memory and then transmit the network delay verification signal including a1 to the remote control center (420). The remote control center (420) can measure the reception time (a2) of the network delay verification signal and record it in the internal memory. The network delay verification response signal including information about the measured reception time (a2) and information about the transmission time (a3) of the network delay verification response signal can be transmitted to the remote control vehicle (410) through a dedicated communication session. Here, the remote control center (420) can store information about the transmission time (a3) of the network delay verification response signal in the internal memory. The delay time measurement module (920) measures the reception time (a4) of the network delay verification response signal, and then calculates the RTT (Round Trip Time) based on the transmission time (a1) of the network delay verification signal, the reception time (a2) of the network delay verification signal, the transmission time (a3) of the network delay verification response signal, and the reception time (a4) of the network delay verification response signal, and divides the calculated RRT by 2 to calculate the network transmission delay time.

Figure 10 shows the results of a network delay measurement experiment according to the present disclosure.

Referring to Fig. 10, network delay measurement can be performed using a Ping Test via TCP communication. A remote control vehicle (410) can transmit a 32-byte sized Ping signal at regular intervals - for example, 64ms TTL (Time To Live) - to a remote control center (420) via a preset TCP communication session.

In this case, the experimental results according to Fig. 10 show that the network delay occurs from a minimum of 1 ms to a maximum of 36 ms, and the average delay for 16 ping transmissions is 8 ms.

The network delay time measurement module (520) can log the network delay measurement results and record them in internal memory.

FIG. 11 is a drawing for explaining the detailed configuration of a validity judgment unit according to an embodiment of the present disclosure.

Referring to FIG. 11, the validity judgment unit (540) may be configured to include a validity judgment signal generation module (1010), a network delay comparison module (1020), a logging module (1030), a threshold setting module (1040), and a control signal interval comparison module (1050).

The network delay comparison module (1020) can verify the validity of the network delay by comparing the measured network delay time with a predetermined network delay threshold. For example, if the measured network delay exceeds the network delay threshold, the network delay comparison module (1020) can determine that the network delay is invalid because it exceeds the allowable range. On the other hand, if the measured network delay is lower than the network delay threshold, the network delay comparison module (1020) can determine that the network delay is valid because it is within the allowable range.

The control signal interval comparison module (1050) can compare the measured control signal reception interval with a threshold range (or threshold value) corresponding to the control signal to determine whether the control signal is valid.

For example, if the measured control signal reception interval is within the corresponding threshold range or is lower than or equal to the corresponding threshold, the control signal interval comparison module (1050) may determine that the corresponding control signal is valid. On the other hand, if the measured control signal reception interval is outside the corresponding threshold range or exceeds the corresponding threshold, the control signal interval comparison module (1050) may determine that the corresponding control signal is invalid.

The network delay comparison module (1020) and the control signal interval comparison module (1050) can each transmit the validity verification results to the validity judgment signal generation module (1010).

The logging module (1030) can log and store the comparison results of the network delay comparison module (1020) and the control signal interval comparison module (1050).

The threshold setting module (1040) can set a network delay threshold and a control signal interval threshold range (or threshold). For example, the threshold setting module (1040) can dynamically select (or determine) and set a threshold range or threshold based on the vehicle's driving status information - for example, driving speed, driving road, etc. - and external environment information - for example, weather information, traffic information, etc.

The validity judgment signal generation module (1010) can generate a validity judgment signal based on the network delay validity judgment result and the control signal validity judgment result.

For example, the validity judgment signal may include, but is not limited to, a first signal for controlling transition from remote driving mode to autonomous driving mode, a second signal for controlling emergency parking of the vehicle, a third signal for controlling generation of a warning alarm, etc.

FIG. 12 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to an embodiment of the present disclosure.

Referring to FIG. 12, when the vehicle enters the remote driving mode, a threshold condition (threshold value or threshold range) for a reception interval for each control signal type can be set (S1101 to S1102). Here, the threshold condition can be set by further considering the vehicle driving state and external environmental variables. For example, the vehicle driving state can include the current driving speed, the type of the road on which it is currently driving, etc. The external environmental variables can include, but are not limited to, the traffic volume of the current driving road, weather, time zone, season, etc.

When a control signal is received, the vehicle can measure the reception time of the received control signal and identify the type (S1103 to S1104).

The vehicle can calculate the reception time interval—i.e., the reception time difference—for two consecutively received control signals of the same type (S1105).

The vehicle can determine whether the calculated reception time interval satisfies a threshold condition (S1106).

As a result of the judgment, if the threshold condition is satisfied, the vehicle can continue to receive control signals for remote driving control.

On the other hand, if the critical condition is not satisfied, the vehicle may determine that the control signal is invalid and perform certain exception processing (S1107). For example, the exception processing may include, but is not limited to, switching from remote driving mode to autonomous driving mode, emergency parking, and outputting a warning alarm.

FIG. 13 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to another embodiment of the present disclosure.

Referring to FIG. 13, when the vehicle enters the remote driving mode, the vehicle may establish a 1:1 dedicated communication session with the remote control center (420) for network delay measurement and set a network delay threshold condition (S1201 to S1202). Here, the threshold condition may be set by further considering the vehicle driving state and external environmental variables. For example, the vehicle driving state may include the current driving speed, the type of the road on which it is currently driving, the type of the area on which it is currently driving (downtown/suburban), etc. The external environmental variables may include, but are not limited to, the traffic volume of the current driving road, weather, time zone, season, etc. For example, the dedicated communication session may be, but is not limited to, a highly reliable TCP session.

The vehicle can generate a network delay validation signal and transmit the generated network delay validation signal to a remote control center (420) through a dedicated communication session (S1203). As an example, a ping signal can be used as the network delay validation signal, but is not limited thereto.

The vehicle can measure the network delay time by receiving a response signal for the network delay validation signal (S1203). Here, the detailed method of measuring the network delay time is replaced with the description of the above-described drawing.

The vehicle can determine whether the measured network delay time satisfies a network delay threshold condition (S1205).

As a result of the judgment, if the network delay threshold condition is satisfied, the vehicle can periodically transmit a network delay validation signal.

On the other hand, if the network delay threshold condition is not satisfied as a result of the judgment in step 1205, the vehicle may determine that the received control signal is invalid and perform certain exception processing (S1206). For example, the exception processing may include, but is not limited to, switching from remote driving mode to autonomous driving mode, emergency parking, and outputting a warning alarm.

FIG. 14 is a flowchart illustrating a method for verifying the validity of a control signal in a vehicle under remote control according to another embodiment of the present disclosure.

Referring to FIG. 14, when the vehicle enters remote driving mode, it can establish a 1:1 dedicated communication session with a remote control center (420) for network delay validation and set network delay threshold conditions (S1301 to S1302).

The vehicle can set control signal reception interval threshold conditions for each control signal type to verify the validity of the control signal reception interval (S1303).

The vehicle can perform a network delay validation procedure and a control signal reception interval validation procedure based on the set threshold conditions (S1304). Here, the network delay validation procedure and the control signal reception interval validation procedure are replaced with the description of the above-described drawing.

The vehicle can determine whether exception handling is necessary based on at least one of the control signal reception interval validation result and the network delay validation result (S1305).

If the judgment result indicates that exception handling is required, the vehicle may perform predefined exception handling (S1306 to S1307). Here, the exception handling may include outputting a warning alarm and/or switching to autonomous driving mode and/or emergency parking.

If the determination is that exception handling is not required, the vehicle may return to step 1304 and continue performing the validation procedure.

The above-described present invention can be implemented as a computer-readable code on a medium in which a program is recorded. The computer-readable medium includes all kinds of recording devices that store data that can be read by a computer system. Examples of the computer-readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and also includes those implemented in the form of a carrier wave (e.g., transmission via the Internet).

Although the ODD providing method of the autonomous vehicle, autonomous driving controller, ODD server, remote control vehicle, remote control center, ODD information transmitter, etc. according to the embodiments of the present invention has been described as a specific embodiment, this is only an example and the present invention is not limited thereto, and should be interpreted as having the widest scope according to the basic idea disclosed in this specification. Those skilled in the art can implement an embodiment that is not specified by combining or replacing the disclosed embodiments, but this also does not exceed the scope of the present invention. In addition, those skilled in the art can easily change or modify the disclosed embodiments based on this specification, and it is clear that such changes or modifications also fall within the scope of the present invention.

Claims (26)

A method for verifying the validity of a control signal in a vehicle that is interconnected with a remote control center via a network,
A step of receiving the control signal while the control signal interval measuring unit is operating in remote control mode;
A step for measuring the interval between receiving the control signal by the control signal interval measuring unit; and
A step in which the validity judgment unit performs the control signal validity verification based on the measured reception interval and performs exception processing based on the control signal validity verification result.
A method comprising: a step of identifying a vehicle device type corresponding to the received control signal; a step of setting a threshold condition based on the identified vehicle device type; and a step of determining the validity of the control signal based on whether the measured reception interval satisfies the threshold condition. delete delete In the first paragraph,
A method wherein the above critical conditions are dynamically set based on at least one of a vehicle driving state and external environmental variables. In paragraph 4,
A method wherein the vehicle driving state includes at least one of a current driving speed and a current driving road type. In paragraph 4,
A method wherein the external environmental variables include at least one of weather, season, and time zone. In the first paragraph,
A method wherein the vehicle device type includes at least one of an accelerator, a braking device, a steering device, a lamp, multimedia, an air conditioning device, a wiper, a heating element, a door lock, a sunroof open/close device, and a window control device. In the first paragraph,
The above critical conditions are maintained and managed in the form of a database or a file in a storage provided in the vehicle. delete In the first paragraph,
The above exception handling is,
An action that outputs a warning alarm;
An action for switching from the above remote control mode to the autonomous driving mode; and
Actions that control emergency parking
A method comprising at least one of: delete In the first paragraph,
A method in which the above control signal is received periodically in the form of a stream, and the reception interval of the control signal is measured based on the difference in reception times between two control signals received consecutively corresponding to the same vehicle device type. A nonvolatile computer-readable storage medium storing at least one computer program comprising instructions that, when executed by at least one processor, cause the at least one processor to perform validation operations on a control signal by interacting with a remote control center over a network,
The above actions are,
A step of receiving the control signal while operating in remote control mode;
A step of measuring the reception interval of the above control signal;
A step of performing validation of the control signal based on the measured reception interval; and
A step for performing exception handling based on the results of the validation of the above control signal.
A storage medium, wherein the step of performing the validation comprises the step of identifying a vehicle device type corresponding to the received control signal, the step of setting a threshold condition based on the identified vehicle device type, and the step of determining the validity of the control signal based on whether the measured reception interval satisfies the threshold condition. For vehicles that are connected to a remote control center via a network,
A remote driving controller that controls remote driving;
Autonomous driving controller that controls autonomous driving;
A transmitter/receiver for receiving control signals while operating in remote control mode; and
A validation device that measures the reception interval of the above control signal and performs validation of the above control signal based on the measured reception interval.
Including,
The validation result for the above control signal is transmitted to at least one of the remote driving controller and the autonomous driving controller, and exception processing is performed.
A vehicle, characterized in that the above validation comprises identifying a vehicle device type corresponding to the received control signal, setting a threshold condition based on the identified vehicle device type, and determining the validity of the control signal based on whether the measured reception interval satisfies the threshold condition. delete delete In Article 14,
A vehicle wherein the above threshold conditions are dynamically set based on at least one of the vehicle driving status and external environmental variables. In Article 17,
A vehicle, wherein the above vehicle driving status includes at least one of a current driving speed and a current driving road type. In Article 17,
A vehicle wherein the above external environmental variables include at least one of weather, season and time zone. In Article 14,
A vehicle device type including at least one of an accelerator, a braking device, a steering device, a lamp, multimedia, an air conditioning device, a wiper, a heating element, a door lock, a sunroof open/close device, and a window control device. In Article 14,
The above critical conditions are maintained and managed in the form of a database or file in a storage provided in the vehicle. delete In Article 14,
The above exception handling is,
Means for outputting a warning alarm to said vehicle;
Means for switching from the above remote control mode to the autonomous driving mode; and
Means for controlling emergency parking
A vehicle comprising at least one of: delete In Article 14,
A vehicle in which the above control signal is received periodically in the form of a stream, and the reception interval of the above control signal is measured based on the difference in reception times between two control signals received consecutively corresponding to the same vehicle device type. In a device mounted in a vehicle and linked to a remote control center via a network,
A validity judgment threshold database that maintains information about the critical conditions required to judge the validity of a control signal;
A control signal interval measuring unit for measuring the interval at which a control signal is received when operating in remote control mode;
A network signal delay measurement unit that sets up a 1:1 dedicated communication session with the remote control center when entering the above remote control mode, and measures the network delay time by transmitting and receiving a signal for network delay verification through the dedicated communication session; and
A validity judgment unit that performs validity verification on the measured reception interval and the measured delay time by referring to the above validity judgment threshold database, and controls that a predetermined exception processing is performed based on the validation result.
Including,
A device in which the validity judgment unit identifies a vehicle device type corresponding to the received control signal, sets the threshold condition based on the identified vehicle device type, and judges the validity of the control signal based on whether the measured reception interval satisfies the threshold condition.