KR102407544B1 - self-driving vehicle remote control system and control method - Google Patents

Abstract

The present invention relates to a remote control system for an autonomous vehicle and a control method therefor. An autonomous vehicle according to an aspect of the present invention; and a control device that forms a network and communicates with the autonomous vehicle, wherein the control device receives a plurality of images photographed by the autonomous vehicle and matches the plurality of images a conversion unit that converts an image from a third-person view; By overlaying route information including a first node necessary for setting the driving route of the autonomous vehicle and a second node used as a reference for setting the driving route on the image of the third person view, visual part to display; a calculation unit configured to perform a calculation operation related to the driving route of the autonomous vehicle; an adjustment unit for remotely controlling the operation of the autonomous vehicle; and a logic unit configured to set a new target path of the autonomous vehicle based on information obtained through at least one of the autonomous vehicle, the converting unit, the visual unit, the calculating unit, and the adjusting unit. and, when a predetermined event related to the autonomous vehicle occurs, authority related to the operation of the autonomous vehicle may be transferred to the control device.

Description

Self-driving vehicle remote control system and control method

The present invention relates to a remote control system for an autonomous vehicle and a control method therefor. Specifically, according to the present invention, the vehicle image is matched and converted into a third-person view, the vehicle path and auxiliary path set on the three-dimensional image, and the expected trajectory of the vehicle, etc. are expressed to the operator, and the trajectory of the vehicle, the path within a finite time , by calculating the escape route and determining the operator's intention (path) from the interface equipment consisting of safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, speed lock and release, and manual/semi-automatic mode change. , to a system and method for remotely controlling an autonomous vehicle.

Since autonomous public transportation vehicles are operated by unmanned or automatic operation, real-time vehicle status information is transmitted to the SCC (Smart Control Center) through smart control (SCC), wired and V2X wireless networks, and control information is transmitted to the vehicle. are doing

Currently, in order to commercialize an autonomous driving service, autonomous driving control in a remote location is being considered in order to supplement the safety problem of an autonomous driving system.

However, remote control has several limitations. First, although mass real-time data communication was possible with the start of the 5G service, there is a risk of slowing down or stopping the transmission speed at the moment of heavy usage within the range of the network. There is a problem with guarantees.

In addition, unlike driving in a real vehicle, visibility of video images transmitted to remote locations is limited due to poor resolution or optical distortion.

In addition, since the operator at a remote location obtains only visual information about the vehicle's movement, the accuracy of steering wheel or acceleration/deceleration pedal operation is deteriorated.

Furthermore, since the moment when control is transferred to an operator in a remote location is an incomplete situation where autonomous driving is generally not possible, the person to whom control has been transferred must quickly recognize the situation, but for the above reasons, it experiences a greater cognitive load than the driver of the vehicle. .

In addition, an operator in a remote location may also have a problem of reduced concentration during long-time driving like a driver of a real vehicle.

In order to solve this problem, the remote control of the autonomous vehicle requires a method to minimize the cognitive load of the operator in a remote location and to minimize the inaccurate control result due to insufficient sense.

Korean Intellectual Property Office Registration No. 10-1634451 Korean Intellectual Property Office Registration No. 10-1002182

An object of the present invention is to solve the above-mentioned problems by providing a user with a remote control system for an autonomous vehicle and a method for controlling the same.

Specifically, according to the present invention, the vehicle image is matched and converted into a third-person view, the vehicle path and auxiliary path set on the three-dimensional image, and the expected trajectory of the vehicle, etc. are expressed to the operator, and the trajectory of the vehicle, the path within a finite time , by calculating the escape route and determining the operator's intention (path) from the interface equipment consisting of safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, speed lock and release, and manual/semi-automatic mode change. , a system and method for remotely controlling an autonomous vehicle are proposed in this specification.

On the other hand, the technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clearly to those of ordinary skill in the art to which the present invention belongs from the description below. can be understood

An autonomous driving vehicle according to an aspect of the present invention for achieving the above technical problem; and a control device that forms a network and communicates with the autonomous vehicle, wherein the control device receives a plurality of images photographed by the autonomous vehicle and matches the plurality of images a conversion unit that converts an image from a third-person view; By overlaying route information including a first node necessary for setting the driving route of the autonomous vehicle and a second node used as a reference for setting the driving route on the image of the third person view, visual part to display; a calculation unit configured to perform a calculation operation related to the driving route of the autonomous vehicle; an adjustment unit for remotely controlling the operation of the autonomous vehicle; and a logic unit configured to set a new target path of the autonomous vehicle based on information obtained through at least one of the autonomous vehicle, the converting unit, the visual unit, the calculating unit, and the adjusting unit. and, when a predetermined event related to the autonomous vehicle occurs, authority related to the operation of the autonomous vehicle may be transferred to the control device.

Also, the plurality of images may be acquired based on a plurality of cameras and lidar devices provided in the autonomous vehicle.

Also, the autonomous vehicle may be a public transportation vehicle, and the first node may be associated with route data of the public transportation.

In addition, the visual unit may additionally display information related to a movement of a vehicle around the autonomous driving vehicle received from the autonomous driving vehicle in a state in which the route information is displayed on the image of the third person view.

In addition, the visual unit may additionally display the current path and movement trajectory information of the autonomous vehicle on the image of the third-person view while the path information is displayed on the image of the third-person view.

Also, the calculator may perform a calculation operation related to at least one of movement trajectory information of the autonomous vehicle, movement route information of the autonomous vehicle within a predetermined period, and information for escaping from the operation route.

In addition, the adjustment unit, at least one of safety distance reduction, lateral path shift, speed deceleration, speed acceleration, node change on the travel route, route reset, speed fix and release, and manual/semi-automatic change related to the autonomous vehicle Functions can be controlled remotely.

Also, the first node means an end point of a road that meets an intersection, and the first node may be designated for each lane.

In addition, when the lateral path shift function is applied, the visual unit additionally displays a virtual path shifted left and right from the current path of the autonomous vehicle on the image of the third person view, and The virtual path may be displayed in a color different from path information previously displayed on the image of the third person view.

In addition, the speed deceleration and speed acceleration functions may be used when a lane change of the autonomous vehicle is required.

Also, the node change function on the driving route may be used in a situation for following the route information and a lane change situation.

In addition, the route reset function may include, when the autonomous vehicle deviates from the route information, resets a route to a first node closest to the current location of the autonomous vehicle among a plurality of first nodes on the route information can be used for

In addition, the speed fixing function is used to fix the reduced speed after reducing the speed of the autonomous vehicle to a predetermined value or less, and in a state in which the speed fixing function is set, the speed deceleration function is activated but the speed acceleration function It is deactivated, and the release function may be used when releasing the speed lock.

Also, the manual/semi-automatic change function may be used to manually move the autonomous vehicle through remote control when semi-automatic driving of the autonomous vehicle is impossible.

In addition, when the obstacle exists in the route information of the autonomous vehicle, the calculator performs an escape route-related calculation so that the autonomous vehicle avoids the obstacle, and the autonomous vehicle operates according to the escape route. and the autonomous vehicle may be driven to the final destination on the route information.

In addition, the safety distance of the autonomous vehicle can be set through the control unit, and the safety distance is set to a greater value among the distance required by the minimum effective trajectory and the distance that is increased or decreased according to the speed of the autonomous vehicle. can be

On the other hand, another aspect of the present invention for achieving the above technical problem is an autonomous vehicle; and a control device that forms a network and communicates with the autonomous driving vehicle, wherein a conversion unit of the control device receives a plurality of images captured by the autonomous driving vehicle, and receives the plurality of received images. A first step of converting the image into a third-person view by matching the image; The visual unit of the control device displays route information including a first node necessary for setting a driving route of the autonomous vehicle and a second node used as a reference for setting the driving route on the image of the third person view. a second step of overlaying and displaying the above; a third step of performing, by a calculation unit of the control device, a calculation operation related to the driving route of the autonomous vehicle; a fourth step of remotely controlling the operation of the autonomous vehicle by a control unit of the control device; and a fifth step of setting, by the logic unit of the control device, a new target path of the autonomous vehicle based on information obtained through at least one of the autonomous vehicle, the conversion unit, the visual unit, the calculation unit, and the adjusting unit; and, when a predetermined event related to the autonomous vehicle occurs, authority related to the operation of the autonomous vehicle may be transferred to the control device.

Also, the autonomous vehicle may be a public transportation vehicle, and the first node may be associated with route data of the public transportation.

Also, in the second step, while the route information is displayed on the image of the third person view, information related to the movement of vehicles around the autonomous driving vehicle received from the autonomous driving vehicle may be additionally displayed.

Also, in the second step, while the route information is displayed on the image of the third person, the current route and movement trajectory information of the autonomous vehicle may be additionally displayed on the image of the third person.

In addition, in the third step, the calculation unit performs a calculation operation related to at least one of movement trajectory information of the autonomous vehicle, movement route information of the autonomous vehicle within a predetermined period, and information for escaping from the operation route. can be done

In addition, in the fourth step, the adjustment unit is configured to reduce the safety distance associated with the autonomous vehicle, shift the lateral path, decelerate the speed, accelerate the speed, change the node on the travel route, reset the route, fix and release the speed, manually/ At least one function of the semi-automatic change may be controlled remotely.

Also, the first node means an end point of a road that meets an intersection, and the first node may be designated for each lane.

Further, in the fifth step, when the obstacle exists in the path information of the autonomous vehicle, the calculator performs an escape route-related calculation so that the autonomous vehicle avoids the obstacle, and according to the escape route, The autonomous vehicle is driven, and the autonomous vehicle may be driven to the final destination on the route information.

In addition, in the fourth step, a safety distance of the autonomous vehicle can be set through the control unit, and the safety distance is a distance required by a change distance that is increased or decreased according to the speed of the autonomous vehicle and a minimum effective escape trajectory. It can be set to a larger value.

The present invention may provide a remote control system for an autonomous vehicle and a method for controlling the same to a user.

Specifically, according to the present invention, the vehicle image is matched and converted into a third-person view, the vehicle path and auxiliary path set on the three-dimensional image, and the expected trajectory of the vehicle, etc. are expressed to the operator, and the trajectory of the vehicle, the path within a finite time , by calculating the escape route and determining the operator's intention (path) from the interface equipment consisting of safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, speed lock and release, and manual/semi-automatic mode change. , a system and method for remotely controlling an autonomous vehicle may be provided to a user.

In addition, according to the present invention, it is possible to minimize the cognitive load of the operator by performing the control in the third person view.

In addition, since the operator according to the present invention transmits the intention of driving and the vehicle performs actual control within the possible range, it is possible to prevent the operator's operation error from leading to an accident.

In addition, according to the present invention, in most downtown driving situations, the operator can minimize fatigue by performing control using the autonomous driving system.

In addition, according to the present invention, since there is no need to move the cockpit of the autonomous vehicle to a remote location, a smaller space and cost can be reduced.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be able

1 is a view for explaining a conventional remote control method.
2 shows an example of a block diagram of an autonomous vehicle remote control system according to the present invention.
3 shows an example of a block diagram related to the autonomous vehicle remote control system according to the present invention.
4 shows an example of a block diagram for explaining the configuration of smart control according to the present invention.
5 shows an example of a control unit proposed by the present invention.
6 shows an example of a vehicle trajectory displayed according to the present invention.
7 shows an example of a main node change scenario according to the present invention.
8 and 9 show an example of a lane change time change scenario according to the present invention.
10 illustrates an example of an illegal parking and stopping scenario in an alley according to the present invention.
Fig. 11 shows an example of a lateral path shift scenario when space is sufficient according to the present invention.
12 shows an example of a lateral path shift scenario in case of lack of space according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. This is not intended to limit the present invention to specific embodiments, it can be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

In describing the present invention, terms such as first, second, etc. may be used to describe various components, but the components may not be limited by the terms. The above terms are only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When a component is referred to as being connected or connected to another component, it may be directly connected or connected to the other component, but it can be understood that other components may exist in between. . On the other hand, when it is mentioned that a certain element is directly connected to or directly connected to another element, it may be understood that another element does not exist in the middle.

The terminology used herein is used only to describe specific embodiments, and is not intended to limit the present invention. The singular expression may include the plural expression unless the context clearly dictates otherwise.

In this specification, the terms include or include are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and includes one or more other features or numbers, It may be understood that the existence or addition of steps, operations, components, parts, or combinations thereof is not precluded in advance.

In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. . Terms such as those defined in a commonly used dictionary may be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present specification, it is interpreted in an ideal or excessively formal meaning. it may not be

Problems of the prior art and object of the present invention

An autonomous vehicle is a vehicle that detects the situation with various sensors of the vehicle, such as precise maps and GPS, and finds its own destination without the driver operating the steering wheel, accelerator pedal, or brake.

The autonomous driving market is expected to enter full-fledged growth from 2020.

According to market research firm Navigant Research, the global autonomous vehicle market is expected to reach $1.2 trillion by 2035 after accounting for $200 billion, or 2%, of the total car market in 2020.

A number of technologies may be required to realize autonomous vehicles, such as HDA technology that automatically maintains the distance between vehicles, Lane Departure Warning System (LDWS), Lane Keeping Assist System (LKAS), Blind Spot Warning System (BSD), Advanced Smart Cruise Control (ASCC) and Automatic Emergency Braking System (AEB) are required.

Since autonomous public transportation vehicles are operated by unmanned or automatic operation, real-time vehicle status information is transmitted to the SCC (Smart Control Center) through smart control (SCC), wired and V2X wireless networks, and control information is transmitted to the vehicle. are doing

Currently, in order to commercialize an autonomous driving service, autonomous driving control in a remote location is being considered in order to supplement the safety problem of an autonomous driving system.

1 is a view for explaining a conventional remote control method.

As shown in (a) and (b) of Fig. 1, the general remote control method of an autonomous vehicle acquires an image from a camera attached to the actual vehicle as follows and sees it from a remote location through a control interface configured in the same way as the vehicle to perform control.

In order to solve the problem of real-time guarantee, a method of solving the safety problem by using a plurality of individual cellular networks is being considered.

In addition, in order to provide the same viewing angle as the actual driver's seat, a method of using a dome-shaped screen or an HMD-type VR has been proposed.

However, remote control has several limitations. First, although mass real-time data communication was possible with the start of the 5G service, there is a risk of slowing down or stopping the transmission speed at the moment of heavy usage within the range of the network. There is a problem with guarantees.

In addition, unlike driving in a real vehicle, visibility of video images transmitted to remote locations is limited due to poor resolution or optical distortion.

In addition, since the operator at a remote location obtains only visual information about the vehicle's movement, the accuracy of steering wheel or acceleration/deceleration pedal operation is deteriorated.

Furthermore, since the moment when control is transferred to an operator in a remote location is an incomplete situation where autonomous driving is generally not possible, the person to whom control has been transferred must quickly recognize the situation, but for the above reasons, it experiences a greater cognitive load than the driver of the vehicle. .

In addition, an operator in a remote location may also have a problem of reduced concentration during long-time driving like a driver of a real vehicle.

That is, the LTE-based video image delay is less than 100ms, which is a good level, but there is a problem that the operator does not feel a sense of heterogeneity in real-time control.

In addition, there is a problem in that the existing operator equipment of the same type requires an excessively large space and at the same time has to transmit an image to a plurality of arranged displays, so that there is a high possibility that a larger delay time is generated.

In addition, there is a problem in that configuring an actual driver's seat at a remote location may cause an erroneous operation of the operator in the current situation in which the sense of the real driver cannot be completely simulated.

Accordingly, the present invention intends to solve the above-mentioned problems by providing a user with a remote control system for an autonomous vehicle and a method for controlling the same.

Specifically, according to the present invention, the vehicle image is matched and converted into a third-person view, the vehicle path and auxiliary path set on the three-dimensional image, and the expected trajectory of the vehicle, etc. are expressed to the operator, and the trajectory of the vehicle, the path within a finite time , by calculating the escape route and determining the operator's intention (path) from the interface equipment consisting of safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, speed lock and release, and manual/semi-automatic mode change. , a system and method for remotely controlling an autonomous vehicle are proposed in this specification.

Prior to the detailed description of the present invention, an autonomous driving vehicle control system applied to the present invention will be described with reference to the drawings.

Autonomous vehicle control system

2 shows an example of a block diagram of an autonomous vehicle control system according to the present invention.

Referring to FIG. 2 , the autonomous vehicle control system 10 according to the present invention includes a smart control (SCC, Smart Control Center, 200), a base station (roadside base station, Road Side Equipment, RSE, 300) and an autonomous vehicle 100 ) may be included.

Here, the smart control 200 may communicate with the base station 300 or directly communicate with the autonomous vehicle 100 .

In general, a plurality of base stations 300 may be provided, and each base station 300 takes charge of each cell area among a plurality of cells and enters the cell area according to the coverage that each base station 300 can communicate with. It is possible to communicate with the autonomous vehicle 100 .

At this time, the plurality of base stations 300 may transmit information received from the autonomous vehicle 100 that has entered each base station 300 to the smart control 200 , and the smart control 200 includes the plurality of base stations 300 . ), it is possible to determine the state of the autonomous vehicle 100 by using the information received from it, and to deliver an action appropriate to the situation.

Data can be exchanged between smart control (SCC, Smart Control Center, 200) and base station (roadside base station, Road Side Equipment, RSE, 300) through wired communication, wireless communication, and mixed wired and wireless communication.

Data may be exchanged between the base station (roadside base station, Road Side Equipment, RSE, 300) and the autonomous vehicle 100 through short-distance communication or long-distance communication.

Typically, short-distance communication may include technologies such as ANT, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), and ZigBee.

In addition, long-distance communication is V2X (Vehicle to Everything) communication environment WAVE (Wireless Access in Vehicular Environments), LTE (Long Term Evolution), WAVE+LTE (hybrid type), CDMA (code division multiple access), FDMA (frequency division) multiple access), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), and single carrier frequency division multiple access (SC-FDMA).

Meanwhile, FIG. 3 shows an example of a block diagram related to an autonomous vehicle control system according to the present invention.

The configuration of FIG. 3 is the same as that of FIG. 2 , and a detailed configuration of the autonomous vehicle 100 according to the present invention will be described.

Referring to FIG. 3 , the autonomous vehicle 100 includes a wireless communication unit 110 , an A/V (Audio/Video) input unit 120 , a user input unit 130 , a sensing unit 140 , an output unit 150 , It may include a memory 160 , an interface unit 170 , a control unit 180 , a power supply unit 190 , and the like.

However, since the components shown in FIG. 3 are not essential, smart control having more or fewer components may be implemented.

Hereinafter, the components will be described in turn.

The wireless communication unit 110 may include one or more modules that enable smart control and wireless communication between the base station 300 and the autonomous vehicle 100, a wireless communication system, or between a device and a network in which the device is located.

For example, the wireless communication unit 110 may include a broadcast reception module 111 , a mobile communication module 112 , a wireless Internet module 113 , a short-range communication module 114 , and a location information module 115 , etc. .

The broadcast reception module 111 receives a driving safety signal, driving safety-related information, a broadcast signal, and broadcasting-related information from an external broadcast management server through a broadcasting channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server is a server that generates and transmits a driving safety signal, driving safety-related information, broadcast signal and broadcast-related information, or receives a pre-generated driving safety signal, driving safety-related information, broadcast signal and broadcast-related information It may mean a server that transmits to the control.

The driving safety signal may include not only a road accident signal, a road damage signal, and an obstacle signal of a planned driving route, but also a driving safety signal in a form in which a driving safety signal is combined on a navigation driving route.

The driving safety-related information may mean information related to a road construction section, a road congestion section, real-time disaster broadcasting, or a driving safety service provider. The driving safety-related information may also be provided through a mobile communication network.

The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a broadcast signal in which a data broadcast signal is combined with a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112 .

The driving safety or broadcasting related information may exist in various forms. For example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast reception module 111, for example, DMB-T (Digital Multimedia Broadcasting-Terrestrial), DMB-S (Digital Multimedia Broadcasting-Satellite), MediaFLO (Media Forward Link Only), DVB-H (Digital Video Broadcast) -Handheld) and ISDB-T (Integrated Services Digital Broadcast-Terrestrial), etc. can be used to receive a digital broadcasting signal using a digital broadcasting system. Of course, the broadcast reception module 111 may be configured to be suitable for other broadcast systems as well as the aforementioned digital broadcast system.

A broadcast signal and/or broadcast related information received through the broadcast reception module 111 may be stored in the memory 160 .

The mobile communication module 112 transmits/receives wireless signals to and from at least one of a base station, an external device, and a server on a mobile communication network.

It may include various types of data according to text/multimedia message transmission/reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built-in or external to smart control. As wireless Internet technologies, wireless LAN (WLAN) (Wi-Fi), wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), etc. may be used.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, RFID (Radio Frequency Identification), infrared data association (IrDA), UWB (Ultra Wideband), ZigBee, and Wi-Fi (Wireless Fidelity, Wi-Fi) are short-range communication technologies. can be used

The location information module 115 is a module for acquiring the location of the smart control, and a representative example thereof is a Global Positioning System (GPS) module.

Referring to FIG. 3 , the A/V (Audio/Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122 . The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a photographing mode. The processed image frame may be displayed on the display unit 151 .

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110 . Two or more cameras 121 may be provided according to the use environment.

The microphone 122 receives an external sound signal by a microphone in a recording mode, a voice recognition mode, and the like, and processes it as electrical voice data. The processed voice data may be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. Various noise removal algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122 .

The user input unit 130 generates input data for the user to control the operation of the smart control. The user input unit 130 may include a keypad, a dome switch, a touch pad (static pressure/capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 is for controlling the operation of smart control by sensing the current state of smart control, such as the opening/closing state of the smart control, the location of the smart control, the presence or absence of user contact, the direction of the smart control, acceleration/deceleration of the smart control, etc. Generates a sensing signal.

The sensing unit 140 may sense whether power is supplied from the power supply unit 190 , whether the interface unit 170 is coupled to an external device, and the like.

Meanwhile, the sensing unit 140 may include a proximity sensor 141 .

The output unit 150 is for generating an output related to sight, hearing or tactile sense, and includes a display unit 151, a sound output module 152, an alarm unit 153, a haptic module 154, and a projector module ( 155) may be included.

The display unit 151 displays (outputs) information processed in smart control.

The display unit 151 is a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display (flexible). display) and at least one of a three-dimensional display (3D display).

Some of these displays may be configured as a transparent type or a light-transmitting type so that the outside can be viewed through them. This may be referred to as a transparent display, and a representative example of the transparent display is a TOLED (Transparant OLED). The rear structure of the display unit 151 may also be configured as a light transmission type structure. With this structure, the user can see an object located at the rear of the smart control body through the area occupied by the display unit 151 of the smart control body.

Two or more display units 151 may exist according to an implementation form of smart control. For example, in smart control, a plurality of display units may be spaced apart or integrally disposed on one surface, or may be respectively disposed on different surfaces.

When the display unit 151 and the sensor for sensing a touch operation (hereinafter, referred to as a 'touch sensor') form a layered structure (hereinafter referred to as a 'touch screen'), the display unit 151 is provided in addition to the output device. It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, and a touch pad.

The touch sensor may be configured to convert a change in pressure applied to a specific part of the display unit 151 or capacitance generated in a specific part of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also the pressure at the time of the touch.

When there is a touch input to the touch sensor, a signal(s) corresponding thereto is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 180 . Accordingly, the controller 180 can know which area of the display unit 151 has been touched, and the like.

The proximity sensor 141 may be disposed in an inner region of the smart control covered by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing in the vicinity without mechanical contact using the force of an electromagnetic field or infrared rays. Proximity sensors have a longer lifespan than contact sensors and their utility is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. When the touch screen is of a capacitive type, it is configured to detect the proximity of the pointer by a change in an electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of description, the act of bringing the pointer closer to the touch screen without making contact so that the pointer is recognized as being positioned on the touch screen is referred to as a “proximity touch”, and the touch The act of actually touching the pointer on the screen is called "contact touch". The position at which a proximity touch is performed with the pointer on the touch screen means a position at which the pointer vertically corresponds to the touch screen when the pointer is touched in proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and the proximity touch pattern may be output on the touch screen.

In addition, the sensing unit 140 according to the present invention may include a visible light image sensor 142 , a lidar sensor 143 , an invisible light image sensor 144 , and a radar sensor 145 .

First, an image sensor is a device that detects subject information and converts it into an electrical image signal, and can be broadly divided into an image pickup tube and a solid-state image sensor. semiconductors (MOS), charge coupled devices (CCDs), and the like.

On the other hand, a visible light image sensor 142 using visible light that can be seen by the human eye and an invisible light image sensor 144 that converts an invisible image such as an ultraviolet region that cannot be seen by the human eye into a visible image are provided in the present invention. can be applied.

In addition, the LIDAR sensor 143 is a technology capable of sensing a distance from an object and various physical properties by illuminating a laser on a target. It can detect surrounding objects and topographical features and model them as 3D images. As self-driving cars become a reality, the LIDAR sensor 143 is attracting attention as a technology that will serve as the eyes of autonomous vehicles.

In addition, the radar sensor 145 is a device that emits a strong electromagnetic wave and receives the echo wave that is reflected from the target object to identify the object or detect the position, moving speed, etc. of the object.

Meanwhile, the sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a recording mode, a voice recognition mode, and a broadcast reception mode. The sound output module 152 also outputs a sound signal related to a function performed in smart control. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of a smart control event.

The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than a video signal or an audio signal, for example, vibration.

The video signal or audio signal may also be output through the display unit 151 or the audio output module 152 , and thus they may be classified as a part of the alarm unit 153 .

The haptic module 154 generates various tactile effects that the user can feel. A representative example of the tactile effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 are controllable.

For example, different vibrations may be synthesized and outputted or output sequentially.

In addition to vibration, the haptic module 154 is a pin arrangement that moves vertically with respect to the contact skin surface, the blowing force or suction force of air through the nozzle or suction port, the grazing on the skin surface, contact with the electrode, electrostatic force, etc. Various tactile effects can be generated, such as the effect of heat absorption and the effect of reproducing a feeling of coolness and warmth using an element capable of absorbing heat or generating heat.

The haptic module 154 may not only transmit the tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through a muscle sense such as a finger or arm. Two or more haptic modules 154 may be provided according to the configuration aspect of the smart control.

The projector module 155 is a component for performing an image project function using smart control, and is the same as or at least part of an image displayed on the display unit 151 according to a control signal of the control unit 180 . can display different images on an external screen or wall.

Specifically, the projector module 155 generates an image to be output to the outside using a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, and light generated by the light source It may include an image generating means (not shown) for doing this, and a lens (not shown) for magnifying and outputting the image to the outside at a predetermined focal length. Also, the projector module 155 may include a device (not shown) capable of adjusting an image projection direction by mechanically moving a lens or the entire module.

The projector module 155 may be divided into a cathode ray tube (CRT) module, a liquid crystal display (LCD) module, a digital light processing (DLP) module, and the like according to the device type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated when light generated from a light source is reflected by a digital micromirror device (DMD) chip.

Preferably, the projector module 155 may be provided on the side, front or rear side of the smart control in the longitudinal direction. Of course, it is natural that the projector module 155 may be provided at any location in the smart control, if necessary.

The memory unit 160 may store a program for processing and control of the controller 180, and has a function for temporarily storing input/output data (eg, message, audio, still image, moving image, etc.) can also be performed. The frequency of use of each of the data may also be stored in the memory unit 160 . In addition, the memory unit 160 may store data related to vibrations and sounds of various patterns output when a touch input on the touch screen is input.

The memory 160 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic It may include at least one type of storage medium among a disk and an optical disk. The smart control may operate in relation to a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a passage with all external devices connected to the smart control. The interface unit 170 receives data from an external device, receives power and transmits it to each component inside the smart control, or allows data inside the smart control to be transmitted to an external device. For example, wired/wireless headset ports, external charger ports, wired/wireless data ports, memory card ports, ports for connecting devices equipped with identification modules, audio input/output (I/O) ports, A video I/O (Input/Output) port, an earphone port, etc. may be included in the interface unit 170 .

The identification module is a chip that stores various information for authenticating the right to use the smart control. Module, USIM) and the like. A device equipped with an identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device can be connected to the smart control through the port.

The interface unit is a path through which power from the cradle is supplied to the smart control when smart control is connected to an external cradle, or a path through which various command signals input from the cradle by the user are transmitted to the mobile device. can be Various command signals or the power input from the cradle may be operated as signals for recognizing that the mobile device is correctly mounted on the cradle.

The controller 180 generally controls the overall operation of the smart control.

The controller 180 may monitor status information of each autonomous vehicle operating in an area controlled by the smart control in real time, and may perform control information on an event that occurs.

The controller 180 may perform a pattern recognition process capable of recognizing a handwriting input or a drawing input performed on the touch screen as characters and images, respectively.

The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented within the controller 180 or may be implemented separately from the controller 180 .

The controller 180 may perform a pattern recognition process capable of recognizing a handwriting input or a drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the control unit 180 to supply power required for operation of each component.

Various embodiments described herein may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays, It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. The described embodiments may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. The software code may be implemented as a software application written in a suitable programming language. The software code may be stored in the memory 160 and executed by the controller 180 .

smart control system

4 shows an example of a block diagram for explaining the configuration of smart control according to the present invention.

Referring to FIG. 4 , the smart control system 200 according to the present invention includes a conversion unit 210 , a time unit 220 , a calculation unit 230 , a management interface unit 240 , a logic unit 250 , and a control unit. part 260 may be included.

First, the conversion unit 210 according to the present invention provides a function of matching the image of the autonomous vehicle 100 and changing it to a third-person view.

The conversion unit 210 applied to the present invention may change the image to the third-person view of the matching vehicle by using a plurality of cameras and a lidar device attached to the vehicle.

In addition, the visual unit 220 according to the present invention, based on the route data divided into the general node and the main node, the route of the vehicle, the vehicle route (main route) set on the three-dimensional image, a route expressed as a shift, etc. (auxiliary route) and the expected trajectory of the vehicle are provided with a visualization function to express to the operator.

Here, the route data includes the route data of public transportation and can be divided into main nodes (connected to the intersection) that must be passed to keep the route and general nodes that are reference routes.

In addition, the visual unit 220 may visually express the movement of the surrounding vehicle in order to reduce the operator's cognitive load on the screen of the third person view.

In addition, the visual unit 220 may visually express the path, trajectory, etc. of the vehicle in order to reduce the operator's cognitive load on the screen of the third person view.

In addition, the calculator 230 provides a function of calculating an escape route as a trajectory of the autonomous vehicle, a route within a finite time, and a main route.

The calculator 230 according to the present invention may calculate an overall route of the vehicle including a safety distance for escaping the vehicle on the main route.

In addition, the management interface unit 240 is composed of safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, speed lock and release, and manual/semi-automatic mode change.

The interface unit 240 is equipment configured to allow an operator to use commands necessary to perform the presented method, and may include a new type of control rod related to the control unit 260 or a type of an existing joystick structure.

In addition, the logic unit 250 provides a function of determining the intention (path) of the operator from the interface unit 240 .

Here, the logic unit 250 may create a new target path of the vehicle based on the content received from the interface device.

In addition, the control unit 260 provides a function of controlling the predicted route based on an algorithm for making the vehicle in a third-person view and the predicted route displayed on the screen.

In this specification, the control unit 260 may be referred to as a control rod.

The joystick 260 includes functions such as changing a main node of an already set route, lateral control, and route resetting.

Here, the main node means the end point of the road that meets the intersection, and is individually designated for each lane. These nodes are points that must be passed in order to reach the destination. If the right end lane must be used at the next intersection to reach the next stop, the node located at the end point of the right end lane is called the main node of the route.

The control stick 260 according to the present invention serves to assist the set path before moving to the fully manual mode.

5 shows an example of a control unit proposed by the present invention.

Referring to FIG. 5 , the operator of the control stick 260 can see the already set route by overlaying it on the road in the third person view. can

In addition, the lateral path shift of the control rod 260 creates a virtual path shifted left and right from the current path and is immediately reflected in the vehicle.

In addition, the lateral path shift of the control rod 260 is a temporary path, and is displayed on the screen in a color different from the existing path overlaid in the third person view.

In addition, the lateral path shift of the control rod 260 may be used for temporarily avoiding illegally parked vehicles or obstacles, or changing a lane at an earlier time than an expected path.

In addition, in FIG. 5 , in the case of safety distance/speed control, the -Y direction basically means deceleration/safe distance increase, and the +Y direction means forced acceleration and minimization of the safety distance.

This safe distance/speed control can be used when a lane change is necessary in a complicated road situation.

In addition, the main node change consists in shifting the main node for following the route of a route preset in public transportation or changing the timing of the lane change.

Since the major node change is route reconfiguration through node change in the vehicle, it may not be reflected immediately in the vehicle and the overlaid route is changed.

In the case of re-route, when the current vehicle deviates from the route, the route is re-established from the current location to the main node.

Also, in the case of speed lock/release, it is a button that fixes the vehicle speed after reducing the speed. However, even in this case, control in the direction of reducing the speed is possible.

In addition, in the case of manual and semi-automatic mode change, it means a change to a fully manual mode for urgently moving the vehicle when semi-automatic driving is impossible.

There may be no restrictions on the control rod 260 as long as it is an interface that distinguishes the above functions.

In addition, the controller of the control stick 260 gives priority to control to prevent collision, and the path the operator sees is a reference value that the control result wants to follow and does not mean an actual path.

In addition, the semi-autonomous driving controller can drive by securing the minimum safe distance to return to the main route at any time.

In addition, if necessary, the safety distance can be forcibly adjusted with the control rod. In this case, additional manipulation may be required to return to the main route.

Meanwhile, FIG. 6 shows an example of a vehicle trajectory displayed according to the present invention.

Referring to FIG. 6 , the controller of the control stick 260 must be able to precisely follow the path, and as a result of the control, provide the operator with the expected trajectory of the vehicle.

The provided trajectory is expressed at a predetermined distance close to each other and at a certain distance from the point where the tracking starts when the proposed route cannot be currently followed.

Such a trajectory can support an operator to check a problem that may occur due to a limitation of the present system that a desired target path may not be followed.

In addition, the controller maintains the safety distance in the longitudinal direction, and the larger value of the distance that is increased or decreased according to the speed and the distance required by the minimum effective escape trajectory can be designated as the safety distance.

In addition, the effective escape trajectory is calculated when there is an obstacle in the current target path, and the safe distance in front through which the vehicle can escape to the main path within the same path can be calculated.

Example 1 - Major Node Change Scenario

7 shows an example of a main node change scenario according to the present invention.

Referring to FIG. 7 , route change control is possible by manual operation.

That is, as shown in FIG. 7 , when it is impossible to enter because there are vehicles stopped at the main node, it is possible to change the main node and drive.

Example 2 - Lane Change Time Change Scenario

8 and 9 show an example of a lane change time change scenario according to the present invention.

As shown in FIGS. 8 and 9 , when it is expected that it is difficult to change a lane on an existing route, the lane change may be performed early by forcibly shifting the lane.

That is, after approaching the target lane by following the newly created yellow lane as much as possible, the route from the current location to the main node can be reset by using the route reset button.

Example 3 - Alley Illegal Parking Scenario

10 illustrates an example of an illegal parking and stopping scenario in an alley according to the present invention.

As shown in FIG. 10 , a yield scenario can be performed using a lateral path shift in a complex alley situation.

In this case, when a path is generated by shifting the lateral path, the controller does not perform control when there is an obstacle in the shifted path as shown in FIG. 10C .

In addition, as shown in (a) of FIG. 10, the controller starts lateral movement only when the target path cannot be followed and the collision does not occur. At this time, the speed is forcibly reduced so that the model-based controller can follow the path more closely. have.

Example 4 - Transverse path shift scenario when space is sufficient

Fig. 11 shows an example of a lateral path shift scenario when space is sufficient according to the present invention.

The result of the lateral path shift control according to space is divided into a case in which the space is sufficient and a case in which the space is not sufficient.

When the space is sufficient, as shown in FIG. 11 , the controller calculates an escape route and controls the vehicle to enter the empty space.

In (b) of FIG. 11 , the vehicle 100 does not move while leaving a safe distance as much as a route necessary for escaping from the front. In this situation, if the shifted path is released, the vehicle returns to the main path.

Example 5- Lateral Path Shift Scenario in Case of Insufficient Space

12 shows an example of a lateral path shift scenario in case of a lack of space according to the present invention.

If the escape route is included due to insufficient space, it may not be possible to follow the final route as shown in Figure 12 (b).

If it is necessary to avoid the oncoming vehicle and attach a possible vehicle, the operator reduces the speed on the -y-axis in the situation (d) of FIG. 12, fixes the speed with the speed lock button, shifts the path in the lateral direction, and then shifts the lever to the +y-axis By reducing the safety distance by manipulating the

In addition, in the case of (e) of FIG. 12, semi-automatic control is no longer possible.

If semi-automatic control is not possible, it can be controlled by entering the fully manual mode, and the z of the steering wheel is controlled in response to the steering angle, the y direction is forward and backward, and the x is assigned the in-phase control value when four-wheel steer is possible. .

Even in the case of fully manual control, the vehicle is set so that the vehicle does not move when a collision is expected in speed control.

That is, based on the conversion unit 210 providing a function of matching the image of the autonomous vehicle 100 to change to a third-person view, and the route data of the vehicle divided into a general node and a main node, a three-dimensional image The visual unit 220 that provides a visualization function for expressing the vehicle path (main path) set on the image, the path (auxiliary path) expressed by shift, etc., and the expected trajectory of the vehicle to the operator may be utilized.

In addition, the calculator 230 provides a function to calculate the trajectory of the autonomous vehicle, the route within a finite time, and the main route, and the safety distance reduction, lateral route shift, speed deceleration, major node change, route reset, It is possible to use the management interface unit 240 composed of speed fixing and releasing, and manual/semi-automatic mode change.

In addition, based on the logic unit 250 providing a function of determining the operator's intention (path) from the interface unit 240 and the algorithm for making the vehicle in a third-person view, the predicted path expressed on the screen A control unit 260 that provides a function to control a path may be utilized.

Through these configurations and scenarios, the operator's cognitive load can be minimized by performing the control from a third-person view, and the operator's operation mistake can be caused by an accident because the operator communicates the intention of driving and the vehicle performs the actual control within the possible range. can be prevented from continuing.

As a result, in most urban driving situations, operators can minimize fatigue by performing control using the autonomous driving system, and since there is no need to move the cockpit of the autonomous vehicle to a remote location, smaller space and cost savings are possible. .

Effects according to the present invention

The present invention relates to an autonomous vehicle control system using cognitive learning data and a control method therefor, and an autonomous vehicle that generates learning data such as lidar, radar, and invisible light image sensor based on recognition technology by a visible light image sensor. A control system and a control method thereof can be provided to a user.

Specifically, in the present invention, the recognition area between heterogeneous sensors is matched, the position and class are derived based on the recognition result of the upper cognitive sensor, the position marking method of the lower cognitive sensor is converted to the upper cognitive sensor method, and the upper cognitive sensor By using the position and class derived from An autonomous vehicle control system using cognitive learning data and a control method thereof may be provided to a user.

According to the present invention, it is necessary to display the position (displayed in box form or pixel unit) and class of an object in a single image in a separate file or format, and since a lot of image data is required for learning, it takes a lot of time and It can solve the problem that cost is necessary.

According to the present invention, it is possible to solve the problem of requiring a lot of time and money by directly labeling the recognition target of each image to generate the conventional learning data.

After all, in the present invention, it is possible to solve many problems, such as a lot of time and money is required to generate the training data, and the quality of the data is affected depending on the operator.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

Meanwhile, the above-described embodiments of the present invention may be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by hardware, the method according to embodiments of the present invention may include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), and Programmable Logic Devices (PLDs). , FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in the memory unit and driven by the processor. The memory unit may be located inside or outside the processor, and may transmit/receive data to and from the processor by various well-known means.

The detailed description of the preferred embodiments of the present invention disclosed as described above is provided to enable any person skilled in the art to make and practice the present invention. Although the above has been described with reference to preferred embodiments of the present invention, it will be understood by those skilled in the art that various modifications and changes can be made to the present invention without departing from the scope of the present invention. For example, those skilled in the art can use each configuration described in the above-described embodiments in a way in combination with each other. Accordingly, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The present invention may be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention. The present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In addition, claims that are not explicitly cited in the claims can be combined to form an embodiment or included as a new claim by amendment after filing.

Claims (25)

autonomous vehicle; and a control device for communicating with the autonomous vehicle by forming a network,
The control device is
a conversion unit that receives a plurality of images captured by the autonomous vehicle and converts the received images into a third-person view by matching the plurality of images;
By overlaying route information including a first node necessary for setting the driving route of the autonomous vehicle and a second node used as a reference for setting the driving route on the image of the third person view, visual part to display;
a calculation unit configured to perform a calculation operation related to the driving route of the autonomous vehicle;
an adjustment unit for remotely controlling the operation of the autonomous vehicle; and
a logic unit configured to set a new target route of the autonomous vehicle based on information obtained through at least one of the autonomous vehicle, the conversion unit, the visual unit, the calculation unit, and the adjusting unit; including,
When a predetermined event related to the autonomous vehicle occurs, the authority related to the operation of the autonomous vehicle is transferred to the control device,

The adjustment unit,
Remotely control at least one of safety distance reduction, lateral route shift, speed deceleration, speed acceleration, node change on the travel route, route reset, speed lock and release, and manual/semi-automatic change related to the autonomous vehicle ,

When the lateral path shift function is applied,
The visual unit additionally displays a virtual path shifted left and right from the current path of the autonomous vehicle on the image of the third person view,
The virtual path shifted left and right is displayed in a color different from the path information previously displayed on the image of the third person view.
The method of claim 1,
The plurality of images is a remote control system for an autonomous vehicle, characterized in that it is acquired based on a plurality of cameras and lidar devices provided in the autonomous vehicle.
The method of claim 1,
The autonomous vehicle is a public transportation vehicle,
The first node is a remote control system for an autonomous vehicle, characterized in that it is related to the route data of the public transportation.
The method of claim 1,
The visual part,
The remote control system of the autonomous driving vehicle, characterized in that in a state in which the route information is displayed on the image of the third person view, information related to the movement of vehicles around the autonomous driving vehicle received from the autonomous driving vehicle is additionally displayed.
The method of claim 1,
The visual part,
In a state in which the route information is displayed on the image of the third-person view, the current route and movement trajectory information of the autonomous vehicle are additionally displayed on the image of the third-person view. .
The method of claim 1,
wherein the calculation unit performs a calculation operation related to at least one of movement trajectory information of the autonomous vehicle, movement route information of the autonomous vehicle within a predetermined period, and information for escaping from the operation route The vehicle's remote control system.
delete The method of claim 1,
The first node means an end point of a road that meets an intersection,
The first node is a remote control system for an autonomous vehicle, characterized in that each lane is designated.
delete The method of claim 1,
The speed deceleration and speed acceleration functions are used when a lane change of the autonomous vehicle is required.
The method of claim 1,
The node change function on the travel route is
The remote control system of an autonomous vehicle, characterized in that it is used in a situation for following the route information and a lane change situation.
The method of claim 1,
The re-route function is
Autonomy, characterized in that it is used to reset a route to a first node closest to the current location of the autonomous vehicle among a plurality of first nodes on the route information when the autonomous vehicle deviates from the route information A remote control system for a driving vehicle.
The method of claim 1,
The speed fixing function is
After reducing the speed of the autonomous vehicle to a predetermined value or less, it is used to fix the reduced speed below the predetermined value,
In the state in which the speed fixing function is set, the speed deceleration function is activated but the speed acceleration function is deactivated,
The remote control system of an autonomous vehicle, characterized in that the release function is used when the speed lock is released.
The method of claim 1,
The manual/semi-automatic change function is,
The remote control system of an autonomous vehicle, characterized in that it is used to manually move the autonomous vehicle through remote control when semi-automatic driving of the autonomous vehicle is impossible.
The method of claim 1,
When an obstacle exists on the route information of the autonomous vehicle,
The calculation unit performs an escape route-related calculation so that the autonomous vehicle avoids the obstacle,
The autonomous vehicle is operated according to the escape route,
The remote control system of an autonomous vehicle, characterized in that the autonomous vehicle is driven in the same way to the final destination on the route information.
The method of claim 1,
Through the adjustment unit, the safety distance of the autonomous vehicle can be set,
The remote control system of an autonomous vehicle, wherein the safety distance is set to a greater value among a change distance that is added or subtracted according to the speed of the autonomous vehicle and a distance required by the minimum effective escape trajectory.
autonomous vehicle; and a control device that forms a network and communicates with the autonomous vehicle;
a first step of receiving a plurality of images captured by the autonomous driving vehicle by a conversion unit of the control device, and converting the received images into a third-person view by matching the plurality of images;
The visual unit of the control device displays route information including a first node necessary for setting a driving route of the autonomous vehicle and a second node used as a reference for setting the driving route on the image of the third person view. a second step of overlaying and displaying the above;
a third step of performing, by a calculation unit of the control device, a calculation operation related to the driving route of the autonomous vehicle;
a fourth step of remotely controlling the operation of the autonomous vehicle by a control unit of the control device; and
a fifth step of setting, by the logic unit of the control device, a new target path of the autonomous vehicle based on information obtained through at least one of the autonomous vehicle, the conversion unit, the visual unit, the calculation unit, and the adjusting unit; do,
When a predetermined event related to the autonomous vehicle occurs, the authority related to the operation of the autonomous vehicle is transferred to the control device,

In the fourth step, the adjustment unit,
Remotely control at least one of safety distance reduction, lateral route shift, speed deceleration, speed acceleration, node change on the travel route, route reset, speed lock and release, and manual/semi-automatic change related to the autonomous vehicle ,

When the lateral path shift function is applied,
The visual unit additionally displays a virtual path shifted left and right from the current path of the autonomous vehicle on the image of the third person view,
The virtual path shifted left and right is displayed in a color different from the path information previously displayed on the image of the third person view.
18. The method of claim 17,
The autonomous vehicle is a public transportation vehicle,
The first node is a remote control method of an autonomous vehicle, characterized in that related to the route data of the public transportation.
18. The method of claim 17,
In the second step,
The remote control method of an autonomous vehicle, characterized in that, in a state in which the route information is displayed on the image of the third-person view, information related to a movement of a vehicle around the autonomous vehicle received from the autonomous vehicle is additionally displayed.
18. The method of claim 17,
In the second step,
In a state in which the route information is displayed on the image of the third-person view, the current route and movement trajectory information of the autonomous vehicle are additionally displayed on the image of the third-person view. .
18. The method of claim 17,
In the third step, the calculation unit,
Remote control of an autonomous vehicle, characterized in that a calculation operation related to at least one of movement trajectory information of the autonomous vehicle, movement route information of the autonomous vehicle within a predetermined period, and information for escaping from the driving route is performed. Way.
delete 18. The method of claim 17,
The first node means an end point of a road that meets an intersection,
The first node is a remote control method of an autonomous vehicle, characterized in that each is designated for each lane.
18. The method of claim 17,
In the fifth step,
When an obstacle exists on the route information of the autonomous vehicle,
The calculation unit performs an escape route-related calculation so that the autonomous vehicle avoids the obstacle,
The autonomous vehicle is operated according to the escape route,
The remote control method of an autonomous vehicle, characterized in that the autonomous vehicle is driven in the same way to the final destination on the route information.
18. The method of claim 17,
In the fourth step, the safety distance of the autonomous vehicle can be set through the adjustment unit,
The remote control method of an autonomous vehicle, characterized in that the safety distance is set to a larger value among a change distance that is increased or decreased according to the speed of the autonomous vehicle and a distance required by a minimum effective escape trajectory.

Images