JP6688090B2 - Object recognition device and object recognition method - Google Patents

Description

  The present invention relates to an object recognition device and an object recognition method.

  BACKGROUND ART Conventionally, there is an object recognition device that recognizes an object from an image captured by a camera provided in a vehicle. As this type of device, there is a device that pre-stores image patterns of a plurality of types of recognition target objects and recognizes the type of the recognition target object by searching the entire image for an area similar to the image pattern (for example, , Patent Document 1).

JP, 2012-27623, A

  However, when the recognition process is performed on the entire image for each type of recognition target object, the processing amount may increase.

  The present invention has been made in view of the above, and an object thereof is to provide an object recognition device and an object recognition method capable of efficiently recognizing a type of a recognition target object.

  In order to solve the above-mentioned problems and achieve the object, an object recognition device according to the present invention includes a region extraction unit and a recognition unit. The area extraction unit extracts a target area in which the shape of the recognition target object exists from the image captured by the imaging device provided in the vehicle. The recognition unit selectively performs recognition processing of the recognition target object on the target region extracted by the region extraction unit out of the entire region of the image.

  According to the present invention, it is possible to efficiently recognize a recognition target object.

FIG. 1 is an explanatory diagram showing an object recognition method by a general device. FIG. 2 is an explanatory diagram showing the object recognition method according to the embodiment. FIG. 3 is a block diagram showing the configuration of the object recognition device according to the embodiment. FIG. 4 is an explanatory diagram showing the extraction processing of the target area by the area extraction unit. FIG. 5 is an explanatory diagram showing dictionary information. FIG. 6 is an explanatory diagram showing priority order information including the priority order of the recognition order. FIG. 7 is an explanatory diagram showing a positional relationship between a vehicle and an entry prohibition at an intersection. FIG. 8 is a flowchart showing a processing procedure of recognition processing executed by the object recognition device according to the embodiment. FIG. 9 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the object recognition device. FIG. 10 is an explanatory diagram showing dictionary information according to the modified example.

  Hereinafter, embodiments of an object recognition device and an object recognition method disclosed in the present application will be described in detail with reference to the accompanying drawings. The present invention is not limited to the embodiments described below.

  An object recognition method according to an embodiment recognizes a shape of a recognition target object from an image obtained by capturing an image of the front of the vehicle by an imaging device provided in the vehicle, and selectively recognizes the recognition target object in a region where the shape exists. It recognizes the type.

  It should be noted that recognition of the recognition target object is not limited to the image of the front of the vehicle and may be recognized from the image of the rear or the side of the vehicle. The imaging device is, for example, a camera included in the drive recorder, a camera module installed in a mobile device such as a smartphone, a digital camera, or the like.

  Further, the recognition target object is a stationary object having a predetermined shape, and includes, for example, a signboard of a restaurant or the like in addition to the above road signs. The vehicle equipped with the object recognition device is, for example, a car such as a private car or a taxi, a motorcycle, a bus, a truck, or the like.

  In the following, first, with reference to FIG. 1, an object recognition method by an apparatus (hereinafter, referred to as a general apparatus) for recognizing an object by a general method from an image captured by an imaging apparatus will be described, and then a diagram will be described. The object recognition method according to the embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an object recognition method by a general device. FIG. 2 is an explanatory diagram showing the object recognition method according to the embodiment.

  Note that, in FIG. 1, a case where a general device recognizes the 40 km speed limit M included in the image P in front of the vehicle captured by the camera shown in FIG. 1A will be described. Further, it is assumed that a general device stores the image patterns of the entry prohibition 100, the 30 km speed limit 101, and the 40 km speed limit 102 shown in (b) to (d) of FIG.

  As shown in FIG. 1A, a general device first acquires an image P obtained by capturing the front of the vehicle from a camera provided in the vehicle. Then, for example, by pattern matching, it is searched from the image P whether the same road sign as the stored image pattern exists.

  Specifically, as shown in (b) of FIG. 1, an image pattern of entry prohibition 100 is used to search for an entry prohibition road sign from the entire image (hatched area). Then, when the image pattern of the entry prohibition 100 is not found, it is determined that there is no entry prohibition road sign in the image P.

  Subsequently, as shown in FIG. 1C, the image pattern of the 30 km speed limit 101 is used to search the entire image for a road sign with the 30 km speed limit similarly. Then, when the image pattern of the 30 km speed limit 101 is not found, it is determined that the image P does not include the road sign of the 30 km speed limit.

  Subsequently, as shown in (d) of FIG. 1, the image pattern of the 40 km speed limit 102 is used to search the entire image similarly for the presence of the road sign of the 40 km speed limit M. Then, when the image pattern of the 40 km speed limit 102 is found, it is determined that the image P includes the road sign of the 40 km speed limit M.

  As described above, in the object recognition method using the general apparatus, the entire image must be repeatedly set as the search range until the road sign with the speed limit M of 40 km is recognized, which increases the processing amount (hatched area). there were.

  Therefore, as shown in FIG. 2, in the object recognition method according to the embodiment, first, an area in which the circle 24, which is the outer shape of the 40 km speed limit M, exists is extracted from the image P acquired from the camera. Then, a recognition process for recognizing the type of road sign is performed only on the extracted region.

  The object recognition apparatus that executes the object recognition method stores a circle 24, an entry prohibition 100a, a 30 km speed limit 101a, and a 40 km speed limit 102a, as shown in (a) to (d) of FIG. And

  The circle 24 is an outer shape common to the entry prohibition 100, the 30 km speed limit 101, and the 40 km speed limit 102 (see (b) to (d) of FIG. 1). Further, the entry prohibition 100a, the 30km speed limit 101a, and the 40km speed limit 102a are image patterns obtained by removing the peripheral portion of the circle 24 which is the outer shape from the entry prohibition 100, the 30km speed limit 101 and the 40km speed limit 102.

  The object recognition method according to the embodiment will be specifically described below. As shown in (a) of FIG. 2, when the object recognition device acquires the image P from the camera, first, the image pattern of the circle 24 is used to search the entire image P (hatched area) for a circle. .

  Then, as shown in FIG. 2B, when the object recognition device finds the image pattern of the circle 24, it extracts the region where the circle exists as the target region 31. The target area 31 is a range to be searched for the type of road sign.

  Then, as shown in FIG. 2B, using the image pattern of the entry prohibition 100a, only the extracted target area 31 is searched for the entry prohibition road sign. Then, when the image pattern of the entry prohibition 100a is not found, it is determined that the entry prohibition road sign does not exist in the image P.

  Subsequently, as shown in FIG. 2C, the image pattern of the 30 km speed limit 101a is used to search for the road sign with the 30 km speed limit similarly from only the target region 31. When the image pattern of the 30 km speed limit 101a is not found, it is determined that the image P does not include the road sign of the 30 km speed limit.

  Then, as shown in FIG. 2D, the image pattern of the 40 km speed limit 102a is used to search for the road sign of the 40 km speed limit M from the target area 31 in the same manner. Then, when an image pattern of the 40 km speed limit 102a is found, it is determined that the image P has a 40 km speed limit M road sign.

  In other words, the object recognition device can intensively search the target area 31 by extracting the target area 31 in which the circle 24 exists from the entire image, and thus the type of road sign can be efficiently recognized. it can.

  The recognition of the shape and type of the road sign is not limited to pattern matching using an image pattern. For example, the dictionary information created by machine learning the image of the road sign is used to identify the road sign. The shape and type may be recognized. Hereinafter, an object recognition device using such machine learning will be described.

  Next, the object recognition device according to the embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the object recognition device 1 according to the embodiment. As shown in FIG. 3, the object recognition device 1 is connected to the camera 2, the traveling state detection device 3, and the in-vehicle device 4.

  The camera 2 is an image pickup device including an image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and is installed at a position where an image of the front of the vehicle is picked up. Then, the image captured by the camera 2 is output to the object recognition device 1.

  The traveling state detection device 3 is, for example, a vehicle speed sensor or a navigation device, and detects the traveling state of the vehicle. The vehicle speed sensor detects the traveling speed of the vehicle by measuring the vehicle speed pulse signal, and outputs it to the object recognition device 1 as vehicle speed information indicating the traveling speed.

  The navigation device also outputs map information, which is electronic map data, and vehicle position information received from a GPS (Global Positioning System) satellite to the object recognition device 1.

  The vehicle-mounted device 4 is, for example, a vehicle-mounted display that displays an image captured by the camera 2, and a frame image surrounding the road sign recognized by the object recognition device 1 may be superimposed on the image of the camera 2 and highlighted. it can. The in-vehicle device 4 is not limited to the display and may be, for example, a drive recorder.

  When the vehicle-mounted device 4 is a drive recorder, the position and type of the road sign recognized by the object recognition device 1 are recorded in association with each other when recording an image captured by a camera included in the drive recorder.

  The object recognition device 1 according to the embodiment includes a control unit 10 and a storage unit 20. First, the storage unit 20 will be described. The storage unit 20 is a storage unit configured by a storage device such as a non-volatile memory or a hard disk drive, and stores priority order information 21 and dictionary information 22.

  The priority order information 21 is information regarding the priority order of the recognition order of the types of road signs, and the order setting unit 14 described later determines the priority order of the recognition order from the priority order information 21 according to the running state of the vehicle. To be done.

  The dictionary information 22 is information serving as a reference for determining whether or not the object appearing in the image P is a road sign, and is the shape information 22a serving as a reference for determining the shape of the road sign, and a reference for determining the type of the road sign. This is information including the type information 22b.

  The shape information 22a and the type information 22b are created in advance by machine learning and stored in the storage unit 20. Here, a procedure for creating the shape information 22a and the type information 22b will be briefly described.

  First, when the shape information 22a is created, a predetermined number (for example, several hundreds to several thousand images) of a road sign image of a predetermined shape (for example, a circle) and an image of an object other than the predetermined shape are used as learning data. prepare. The sizes of the images to be prepared are unified to, for example, 36 pixels × 36 pixels.

  Subsequently, for example, a HOG (Histgram of Oriented Gradient) feature amount is extracted from each prepared image. Then, the prepared image is plotted on a two-dimensional plane based on the extracted HOG feature amount.

  Then, for example, a discriminator such as an SVM (Support Vector Machine) generates a separation line that separates an image of a road sign having a predetermined shape and an image of an object other than the predetermined shape on a two-dimensional plane.

  The information of the coordinate axes of the two-dimensional plane and the separating line generated by the discriminator becomes the shape information 22a used as a criterion for determining whether or not the image included in the image P is a road sign having a predetermined shape. The feature amount extracted from the prepared image is not limited to the HOG feature amount, and may be, for example, a SIFT (Scale Invariant Feature Transform) feature amount. The discriminator used is not limited to SVM, and may be a discriminator such as AdaBoost.

  In addition, when the type information 22b is created, a predetermined number (for example, several hundred to several thousand) of images of a predetermined type of road sign (for example, 40 km speed limit) and images other than this type of road sign are used as learning data. Images) Get ready.

  The feature quantity extracted from the prepared image and the classifier that generates the line that separates the two-dimensional plane are the same as those used when the shape information 22a is generated. Then, the discriminator generates a separation line that separates the image of the road sign of the speed limit of 40 km and the other images on the two-dimensional plane.

  Then, the information of the coordinate axes of the two-dimensional plane and the separation line generated by the discriminator becomes the type information 22b used as a criterion for determining whether or not the image included in the image P is a predetermined type of road sign. The coordinate axes included in the type information 22b are the same regardless of the type of road sign, and the separating line differs depending on the type of road sign.

  Thereby, the area extracting unit 13 and the recognizing unit 15, which will be described later, by using the shape information 22a and the type information 22b that are the dictionary information 22, the shape of the road sign of the image in 36 pixels × 36 pixels in the image P. And its type can be recognized.

  The control unit 10 includes a scaled image generation unit 11, a feature extraction unit 12, a region extraction unit 13, an order setting unit 14, a recognition unit 15, and an output unit 16.

  The control unit 10 is, for example, a microcomputer including a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory). The CPU performs arithmetic processing in accordance with a program stored in advance in the ROM, for example, so that the scaled image generation unit 11, the feature extraction unit 12, the region extraction unit 13, the order setting unit 14, the recognition unit 15, and the output described above. Functions as the unit 16.

  The scaled-image generation unit 11 generates a plurality of scaled-up images and scaled-down images (for example, 25 scaled-up and scaled-down images) having different scaling rates for the image P input from the camera 2, and outputs the scaled-up and scaled-down images to the feature extraction unit 12. .

  The feature extraction unit 12 extracts, for example, a HOG feature amount from the generated scaled image. The feature amount extracted by the feature extraction unit 12 is unified with the feature amount used when the shape information 22a is generated (for example, the HOG feature amount). Then, the feature extraction unit 12 outputs information on the extracted feature amount of each scaled image to the region extraction unit 13.

  The extraction of the feature amount by the feature extraction unit 12 is not limited to the HOG feature amount, and for example, Hough transform may be used. When the Hough transform is used, the area extraction unit 13 described later recognizes the shape of the road sign by pattern matching.

  The area extracting unit 13 extracts the target area 31 in which the shape to be recognized exists based on the feature amount information extracted by the feature extracting unit 12 and the shape information 22a. Here, the extraction processing of the target area 31 by the area extracting unit 13 will be specifically described with reference to FIG. FIG. 4 is an explanatory diagram showing the extraction processing of the target area 31 by the area extracting unit 13.

  As shown in (a) of FIG. 4, the area extracting unit 13 firstly searches a plurality of search areas An (n = 10) for searching the 40 km speed limit M for the entire area of the enlarged / reduced image R having a predetermined enlargement / reduction rate. ˜N. Hereinafter, a plurality of search areas An are collectively described as search area A).

  The size of the set search area A is, for example, 36 pixels × 36 pixels, which is the same as the size of the image prepared for generating the shape information 22a.

  Then, as shown in FIG. 4B, the area extracting unit 13 forms the search area A in two dimensions based on the coordinate axes included in the shape information 22a based on the feature amount extracted from each search area A. Plot on a plane. That is, by aligning the size of the search region A and the size of the image prepared for generating the shape information 22a, it is possible to plot on the two-dimensional plane. The graph shown in (b) of FIG. 4 is assumed to be composed of a vertical axis and a horizontal axis that represent the feature quantity to be extracted.

  Then, the region extraction unit 13 positions the plotted search region A in either the region 40 showing a circle or the region 41 showing a shape other than that among the regions separated by the separation line S of the shape information 22a. To determine.

  Then, as shown in (c) of FIG. 4, the area extracting unit 13 extracts the search area A21 (see (b) of FIG. 4) located in the area 40 showing a circle as the target area 31 in which the circle 24 exists. To do.

  Here, the shape information 22a is information used to determine whether or not the image within 36 pixels × 36 pixels is circular, as described above. The size of the circular road sign in the image P is not limited to 36 pixels × 36 pixels because the size varies depending on the distance from the vehicle.

  For this reason, although the area extraction unit 13 may not be able to extract the target area 31 in which a circle exists from the image P, the area extraction unit 13 searches for a plurality of scaled images having different scale rates input from the scaled image generation unit 11. As a result, the target area 31 can be extracted.

  It should be noted that the position and number of the search area A shown in (a) of FIG. 4 only show a part for facilitating understanding, and actually, a part of the search area A overlaps. By doing so, it is set so that the enlarged / reduced image R can be searched for without exception.

  Then, the region extraction unit 13 outputs to the recognition unit 15 information regarding the enlargement / reduction ratio of the enlarged / reduced image R in which the extracted target region 31 exists and the position of the target region 31 in the enlarged / reduced image R. The information regarding the position is, for example, the XY coordinate value of the position where the target region 31 exists in the scaled image R converted into the XY orthogonal coordinate system.

  The order setting unit 14 sets the priority order of the recognition order of the types of road signs based on the traveling state of the vehicle. Specifically, first, the order setting unit 14 acquires the image P captured by the camera 2 and the traveling state information indicating the traveling state from the traveling state detection device 3.

  Further, the order setting unit 14 acquires the traveling speed of the vehicle, for example, based on the vehicle speed information that is the traveling state information. Further, the road information indicating the type of road on which the vehicle is traveling is acquired based on the map information and the position information which are the traveling state information.

  In addition, the order setting unit 14 detects the lanes that divide the lanes of the road on which the vehicle is traveling based on the image P acquired from the camera 2 to detect the lanes in which the vehicle is traveling and the lanes in which the vehicle is traveling. Information such as whether or not the vehicle has deviated.

  Then, the order setting unit 14 selects the priority order of the recognition order of the types of road signs according to the traveling state of the vehicle from the priority order information 21 based on the acquired various information. Then, the order setting unit 14 outputs information indicating the priority order of the selected recognition order to the recognition unit 15. A specific example of the priority order information 21 will be described later with reference to FIG.

  The recognition unit 15 selectively performs the recognition process of the type of the road sign that is the recognition target object on the target region 31 extracted by the region extraction unit 13 in the entire region of the image P. Specifically, the recognition unit 15 firstly, based on the information regarding the scaling ratio of the scaled image R in which the target region 31 exists and the position of the target region 31 in the scaled image R from the region extraction unit 13, the target region 31. The size and location of the.

  Subsequently, the recognition unit 15 executes a recognition process for recognizing the type of the road sign only for the specified target area 31 based on the type information 22b included in the dictionary information 22. Here, the dictionary information 22 including the type information 22b and the shape information 22a will be specifically described with reference to FIG.

  FIG. 5 is an explanatory diagram showing the dictionary information 22. As shown in FIG. 5, the dictionary information 22 includes items such as “shape”, “marker”, and “name”, for example. The dictionary information 22 shown in FIG. 5 is an example, and the present invention is not limited to this.

  Further, in FIG. 5, the shape of the road sign and the type of the sign are shown as a symbol image for ease of explanation, but in reality, the “shape” and the “sign” shown in FIG. 5 are determined. The information indicating the coordinate axis and the separating line for is stored.

  The "shape" shown in FIG. 5 is the shape information 22a, which is the information of the coordinate axis and the separating line for determining the shape of the road sign. The “sign” is the type information 22b, which is information about the coordinate axis and the separating line for recognizing the type of the road sign having the shape corresponding to the “shape”. “Name” indicates the name of the road sign of the corresponding “sign”.

  That is, the recognition unit 15 extracts the feature amount of the specified target region 31, and plots the target region 31 on the two-dimensional plane formed by the coordinate axes included in the type information 22b based on the feature amount.

  Then, the recognition unit 15 draws a separation line for each type of road signs included in the type information 22b on the two-dimensional plane, so that the plot position of the target area 31 indicates the area indicating the type of the specific road sign and that area. The type of the road sign is recognized by determining in which area the object other than is located.

  In this way, by narrowing down to the target area 31 in which the outer shape common to the road signs exists, the processing amount for recognizing the type of each road sign can be suppressed, so that the type of road sign can be efficiently recognized. it can.

  The recognition unit 15 may also generate a scaled image of the target region 31 having a different scale rate. Then, the feature amount is extracted for each scaled image to recognize the type of road sign. The number of scaled images generated at this time may be smaller than the number generated by the scaled image generation unit 11 described above (for example, three scaled images).

  This is because, for example, if the position of the target area 31 is displaced for some reason, the type of road sign may not be recognized. Therefore, by generating a scaled image of the target area 31, it is possible to improve the recognition accuracy of the types of road signs.

  Further, the recognition unit 15 changes the priority order of the recognition order of the types of road signs, based on the information indicating the priority order of the recognition order input from the order setting unit 14. Here, the priority of the recognition order by the recognition unit 15 will be specifically described with reference to FIG.

  FIG. 6 is an explanatory diagram showing the priority order information 21 including the priority order of the recognition order. Note that FIG. 6 is an example, and the present invention is not limited to this. The recognition unit 15 follows the priority order of the recognition order selected by the order setting unit 14 in the priority order information 21 shown in FIG.

  As shown in FIG. 6, the priority order information 21 includes items such as “road”, “vehicle speed”, and “recognition order”. “Road” is the type of road on which the vehicle is traveling. The "vehicle speed" is the traveling speed of the vehicle. The “recognition order” indicates the order of recognition of road signs by the recognition unit 15.

  That is, the recognition unit 15 changes the priority order of the recognition order of the types of road signs based on the traveling state information such as the type of road on which the vehicle is traveling and the traveling speed of the vehicle. Specifically, as illustrated in FIG. 6, when the order setting unit 14 acquires road information indicating that the road on which the vehicle is traveling is a highway, the recognition unit 15 sets a speed limit of 60 km or more. The types of road signs are recognized in the recognition order in which the priority of is increased.

  In addition, when the road information indicating that the road on which the vehicle is traveling is a general road is acquired, the priority of the speed limit of 60 km or less is increased. That is, the processing amount can be reduced by narrowing down to the types of road signs that are likely to exist according to the type of road on which the vehicle travels.

  Further, when the order setting unit 14 acquires the vehicle speed information indicating that the own vehicle is traveling on the highway at a speed of 80 km / h, the recognition unit 15 first recognizes the 80 km speed limit. Then, the recognition unit 15 recognizes the speed limit of 70 km, the speed limit of 60 km, the speed limit of 60 km, and the entry prohibition in the order close to the speed limit of 80 km.

  This is because the characteristics of the driver often cause the vehicle to travel at a speed close to the speed limit. For example, a driver who normally drives about 10 km slower than the speed limit may preferentially recognize the speed limit 10 km faster than the traveling speed in consideration of such characteristics.

  In this way, by using the traveling speed of the vehicle, it is possible to start from the type of speed limitation that is likely to exist, so that the recognition process can be executed more efficiently.

  Therefore, the recognition unit 15 can reduce the number of times of recognition processing for recognizing the type of road sign by changing the priority order of the recognition order based on the traveling state of the vehicle, and thus recognize the road sign more efficiently. can do.

  Further, for example, in the image (hereinafter, referred to as a frame image) continuously input from the camera 2 at a predetermined interval, the recognition unit 15 determines the type of speed limitation which is the recognition result of the frame image at the time one frame before. You may preferentially recognize.

  In this case, the recognition unit 15 changes the priority order by storing the recognition result in the storage unit 20 and acquiring the recognition result stored in the order setting unit 14 in the frame image at the next time.

  In addition, as shown in FIG. 6, when the order setting unit 14 acquires road information indicating that the vehicle is approaching an intersection or a branch point, the recognition unit 15 gives priority to a road sign prohibiting entry. To recognize.

  In addition, the priority order of the entry prohibition may be changed according to the position where the entry prohibition is provided at the intersection or the branch point. Such a case will be specifically described with reference to FIG. 7.

  FIG. 7 is a diagram showing a positional relationship between a vehicle and an entry prohibition at an intersection. Note that FIG. 7 shows an example of a scene in which a vehicle C equipped with the object recognition device 1 according to the embodiment approaches an intersection where entry prohibitions M1 and M2 exist.

  In addition, FIG. 7 shows a lane 50 and an oncoming lane 51 on which the vehicle C is traveling. When viewed from the vehicle C, an entry prohibition M1 that prohibits a left turn before the intersection on the lane 50 side is shown at the back of the intersection on the opposite lane 51 side. The entry prohibition M2 for prohibiting a right turn is shown in FIG.

  In such a case, there is a high possibility that the driver of the vehicle C overlooks the entry prohibition M1 and turns to the left by mistake. Therefore, the recognition unit 15 causes the region extraction unit 13 to detect the target region 31 in which a circle exists from the lane 50 side. If is extracted, the priority of entry prohibition is raised.

  On the other hand, even if the driver of the vehicle C overlooks the entry prohibition M2, it is unlikely that the driver crosses the oncoming lane 51 and turns right into the entry prohibition M2 road, but the entry prohibition M2 is also prioritized. If the rank is raised, the processing amount may increase.

  Therefore, when the area extraction unit 13 extracts the target area 31 in which a circle exists from the oncoming lane 51 side, the recognition unit 15 lowers the priority level of the entry prohibition. As a result, it is possible to suppress an increase in processing amount due to unnecessary recognition processing.

  The determination whether the position of the target area 31 is on the lane 50 side or on the opposite lane 51 side is based on the information on the lane markings acquired by the order setting unit 14 and the lane markings and the target area in the image. It can be determined by specifying the positional relationship with 31.

  In addition, in the situation of FIG. 7, when the order setting unit 14 acquires the traveling state information indicating that the vehicle C crosses the oncoming lane 51 and turns right, the priority order of entry prohibition may be increased. Such traveling state information includes, for example, determination information on whether or not the vehicle C has deviated from the central lane marking 52, and information on the distance between the vehicle C and the entry prohibition M2.

  That is, when the order setting unit 14 acquires information indicating that the distance between the vehicle C and the entry prohibition M2 is equal to or less than the predetermined distance based on the image of the camera 2, the recognition unit 15 determines the priority of the entry prohibition M2. Try to raise.

  That is, the recognition unit 15 changes the priority of the entry prohibition M2 based on the distance between the vehicle C and the road sign that changes as the vehicle C travels. As a result, the priority of the recognition processing can be increased only when necessary while reducing the unnecessary recognition processing, so that the recognition processing can be executed more efficiently. Then, the recognition unit 15 outputs the recognition result to the output unit 16.

  The output unit 16 converts and outputs the information indicating the recognition result of the recognition unit 15 according to the form of the in-vehicle device 4. For example, when the vehicle-mounted device 4 is a display, a frame image surrounding the recognized road sign is generated, and a composite image in which the image of the camera 2 and the frame image are superimposed is output.

  Further, when the vehicle-mounted device 4 is a drive recorder, the output unit 16 outputs the traveling history information in which the image of the camera 2 is associated with the position and type of the recognized road sign. Then, the drive recorder records the traveling history information, so that the recognition result can be confirmed ex post facto.

  Next, the processing procedure of the recognition processing executed by the object recognition device 1 according to the embodiment will be described with reference to FIG. FIG. 8 is a flowchart showing a processing procedure of recognition processing executed by the object recognition device 1 according to the embodiment.

  As shown in FIG. 8, the scaled image generation unit 11 acquires the image P captured by the camera 2 and generates a scaled image of the image P (step S101). Subsequently, the feature extraction unit 12 extracts the feature amount from each scaled image using, for example, the HOG feature amount (step S102).

  Subsequently, the area extraction unit 13 extracts a target area in which the shape of the recognition target object exists based on the feature amount extracted by the feature extraction unit 12 and the shape information 22a (step S103). Subsequently, the recognition unit 15 selectively performs recognition processing of the recognition target object on the target region 31 extracted by the region extraction unit 13 in the entire region of the image P based on the type information 22b ( (Step S104), the process ends.

  Further, the object recognition device 1 according to the embodiment can be realized by the computer 200 having the configuration illustrated in FIG. 9 as an example. FIG. 9 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the object recognition device 1.

  The computer 200 includes a CPU (Central Processing Unit) 210, a ROM (Read Only Memory) 220, a RAM (Random Access Memory) 230, and an HDD (Hard Disk Drive) 240. The computer 200 also includes a media interface (I / F) 250, a communication interface (I / F) 260, and an input / output interface (I / F) 270.

  The computer 200 may include an SSD (Solid State Drive), and the SSD may execute some or all functions of the HDD 240. An SSD may be provided instead of the HDD 240.

  The CPU 210 operates based on a program stored in at least one of the ROM 220 and the HDD 240, and controls each unit. The ROM 220 stores a boot program executed by the CPU 210 when the computer 200 starts up, a program dependent on the hardware of the computer 200, and the like. The HDD 240 stores programs executed by the CPU 210, data used by the programs, and the like.

  The media I / F 250 reads a program or data stored in the storage medium 280 and provides it to the CPU 210 via the RAM 230. The CPU 210 loads the program from the storage medium 280 onto the RAM 230 via the medium I / F 250 and executes the loaded program. Alternatively, the CPU 210 executes the program using such data. The storage medium 280 is, for example, a magneto-optical recording medium such as a DVD (Digital Versatile Disc), an SD card, or a USB memory.

  The communication I / F 260 receives data from another device via the network 290, sends the data to the CPU 210, and transmits the data generated by the CPU 210 to the other device via the network 290. Alternatively, communication I / F 260 receives a program from another device via network 290, sends the program to CPU 210, and CPU 210 executes the program.

  The CPU 210 controls the vehicle-mounted device 4 such as a display via the input / output I / F 270. The CPU 210 acquires data from the camera 2 and the traveling state detection device 3 via the input / output I / F 270. Further, the CPU 210 outputs the generated data to the in-vehicle device 4 via the input / output I / F 270.

  For example, when the computer 200 functions as the object recognition device 1, the CPU 210 of the computer 200 executes the program loaded in the RAM 230 to execute the enlarged / reduced image generation unit 11, the feature extraction unit 12, the region extraction unit 13, and the order. The respective functions of the setting unit 14, the recognition unit 15, and the output unit 16 are realized.

  The CPU 210 of the computer 200 reads, for example, these programs from the storage medium 280 and executes the programs, but as another example, the programs may be acquired from another device via the network 290. Further, the HDD 240 can store the priority order information 21 and the dictionary information 22 stored in the storage unit 20.

  As described above, the object recognition device 1 according to the embodiment includes the area extraction unit 13 and the recognition unit 15. The area extraction unit 13 extracts the target area 31 in which the shape of the recognition target object exists from the image captured by the camera 2 provided in the vehicle, based on the feature amount extracted by the feature amount extraction unit 12. The recognition unit 15 selectively executes the recognition process of the recognition target object on the target region 31 extracted by the region extraction unit 13 in the entire region of the image P.

  Thereby, the object recognition device 1 according to the embodiment can efficiently recognize the type of the recognition target object.

  In addition, although the road sign is shown as an example of the recognition target object in the above-described embodiment, the present invention is not limited to this. Here, the dictionary information 22 according to the modification will be described with reference to FIG. 10. FIG. 10 is an explanatory diagram of the dictionary information 22 according to the modification.

  Generally, restaurants and the like that exist around the road have a signboard of a predetermined shape installed near the road to show the existence of the shop to the driver, or a signboard that imitates a specific character for the shape. In many cases, the sign can be used to identify the store.

  Therefore, machine learning is performed using the image of the signboard as learning data to generate the dictionary information 22 showing the characteristics of the signboard of the specific store shown in FIG. 10. Then, the area extracting unit 13 extracts the target area 31 in which the shape of the signboard exists.

  Subsequently, the recognition unit 15 performs a recognition process of selectively recognizing the content (type) of the signboard with respect to the target region 31 extracted by the region extraction unit 13 in the entire region of the image P. Thereby, the object recognition device 1 can efficiently perform the recognition process even when the recognition target object is a signboard or the like.

  Further, according to the object recognition device 1 according to the modified example of the embodiment, for example, even if a general rectangular signboard as shown in FIG. It can be recognized as "XX Sushi".

  In addition, if the shape of the signboard has a characteristic shape imitating a specific character, for example, “X”, even if the recognition process of the content of the signboard is simplified, the “X curry Can be recognized.

  The area of the target area 31 described above is not limited to a rectangular area (see FIG. 4) that is in contact with the outer shape, and may be, for example, a rectangular area outside a circle of a road sign by a predetermined distance. Good. Further, the shape of the target area 31 is not limited to a rectangle, and may be a circle or the same shape as the shape of the recognition target object.

  Further, in the above description, the object recognition device 1 and the in-vehicle device 4 are described as different devices, but they may be integrated. That is, one device may have both the functions of the object recognition device 1 and the in-vehicle device 4 described above.

  Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the present invention are not limited to the particular details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

1 Object Recognition Device 2 Camera 11 Scaled Image Generation Unit 12 Feature Extraction Unit 13 Region Extraction Unit 15 Recognition Unit 31 Target Region

Claims (7)

A region extraction unit that extracts a target region in which the shape of a road sign that is a recognition target is present from an image captured by an imaging device provided in a vehicle,
A recognition unit that selectively performs the recognition process of the recognition target object on the target region extracted by the region extraction unit in the entire region of the image ,
The recognition unit is
Acquiring traveling state information indicating the traveling state of the vehicle, changing the priority order of the recognition order of the road signs based on the traveling state information, the outer shape of the road sign is circular, and the road sign is an oncoming lane. An object recognition device, characterized in that when it is located on the side, the priority of the entry prohibition is lowered . The area extraction unit,
Object recognition apparatus according to claim 1, characterized in that to extract the target region in which the outer shape similar to road signs to be the recognition target object is present. Further comprising a storage unit that stores type information indicating the characteristics of the road sign corresponding to the type of the road sign,
The recognition unit is
The object recognition device according to claim 2, wherein the type of the road sign is recognized based on the type information stored in the storage unit. The recognition unit is
The object recognition device according to any one of claims 1 to 3, wherein vehicle speed information indicating a traveling speed of the vehicle is acquired as the traveling state information. The recognition unit is
The object recognition device according to claim 1, wherein road information indicating a type of a road on which the vehicle is traveling is acquired as the traveling state information. The recognition unit is
Based on the distance between the road signs and the vehicle that varies with the running of the vehicle, the object according to any one of claims 1 to 5, characterized in that to change the priority of the entry prohibition Recognition device. A step of extracting a target area in which the shape of the road sign , which is a recognition target, exists from the image captured by the image capturing device provided in the vehicle;
A step of selectively recognizing the recognition target object with respect to the target area extracted by the area extracting section in the entire area of the image;
The recognition unit acquires running state information indicating a running state of the vehicle, changes the priority order of the recognition order of the road signs based on the running state information, and the outer shape of the road sign is circular, When the road sign is located on the side of the oncoming lane, the step of lowering the priority of the entry prohibition is included.

Images