[go: up one dir, main page]

CN117831004B - Method, device, equipment and medium for detecting obstacle of formula car - Google Patents

Method, device, equipment and medium for detecting obstacle of formula car

Info

Publication number
CN117831004B
CN117831004B CN202410140039.7A CN202410140039A CN117831004B CN 117831004 B CN117831004 B CN 117831004B CN 202410140039 A CN202410140039 A CN 202410140039A CN 117831004 B CN117831004 B CN 117831004B
Authority
CN
China
Prior art keywords
cone
dimensional
cone barrel
projection
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410140039.7A
Other languages
Chinese (zh)
Other versions
CN117831004A (en
Inventor
李迎彬
敖银辉
黄晋豪
陈铿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202410140039.7A priority Critical patent/CN117831004B/en
Publication of CN117831004A publication Critical patent/CN117831004A/en
Application granted granted Critical
Publication of CN117831004B publication Critical patent/CN117831004B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Operations Research (AREA)
  • Algebra (AREA)
  • Image Analysis (AREA)

Abstract

本申请涉及一种无人驾驶方程式赛车障碍物检测方法、装置、设备及介质，所述方法包括：基于预训练的锥桶检测模型对赛道场景图像进行目标检测，确定赛道场景图像中各个颜色种类的锥桶的二维框；基于锥桶的二维框假设锥桶的三维框，构建三维框与二维框相关联的投影组合，基于投影组合中的投影点对以构造关联像素坐标和锥桶三维坐标的超定方程组，采用最小二乘法对超定方程组进行求解，确定锥桶在相机坐标系的三维位置坐标；基于三维位置坐标确定各个颜色种类的锥桶相对于无人驾驶方程式赛车的距离，根据距离指示无人驾驶方程式赛车调整行驶方向。本申请能够准确、快速地避免无人驾驶方程式赛车撞击赛道中的锥桶障碍物。

This application relates to a method, apparatus, device, and medium for obstacle detection in an unmanned Formula One racing car. The method comprises: performing target detection on a track scene image based on a pretrained cone detection model, determining a two-dimensional frame of cones of various colors in the track scene image; hypothesizing a three-dimensional frame of the cones based on the two-dimensional frame of the cones, constructing a projection combination associating the three-dimensional frame with the two-dimensional frame; constructing an overdetermined system of equations associating pixel coordinates with the three-dimensional coordinates of the cones based on the projected point pairs in the projection combination; solving the overdetermined system of equations using the least squares method to determine the three-dimensional position coordinates of the cones in the camera coordinate system; determining the distance of the cones of various colors relative to the unmanned Formula One racing car based on the three-dimensional position coordinates, and instructing the unmanned Formula One racing car to adjust its driving direction based on the distance. This application can accurately and quickly prevent the unmanned Formula One racing car from colliding with cone obstacles on the track.

Description

Method, device, equipment and medium for detecting obstacle of formula car
Technical Field
The application relates to the field of unmanned vehicles, in particular to a method for detecting obstacles of equation motorcycle race, a corresponding device, electronic equipment and a computer readable storage medium.
Background
In recent years, with the development of mathematics, sensor technology and computer hardware technology. In this context, environmental awareness is a key technology in unmanned operation, and accuracy, real-time performance and accuracy of target detection all have important influence on overall stability of a subsequent unmanned operation system. The racing track is mainly formed by enclosing a conical barrel, wherein the left side of the racing track is a red conical barrel, and the right side of the racing track is a blue conical barrel. There are also large yellow cones and small yellow cones at the starting and stopping positions of the racing car.
At present, for cone detection on an unmanned equation race track, the position of a cone on a two-dimensional image is judged through colors and approximate shapes, the method can reduce the two-dimensional detection accuracy of the cone and further reduce the three-dimensional detection accuracy, or the algorithm can not eliminate the influence of simultaneous projection of background point cloud and cone point cloud into a two-dimensional detection frame by utilizing projection of laser point cloud to the two-dimensional detection to realize the perception of the cone position, and the three-dimensional position of the cone is obtained by multi-line laser radar or information fusion of a camera and a radar, so that the hardware cost is high.
In summary, the method is suitable for the problems of low accuracy of two-dimensional detection of the cone barrel, reduced accuracy of three-dimensional detection, high hardware cost and the like of cone barrel detection on an unmanned equation track in the prior art, which are obtained by multi-line laser radar or information fusion of a camera and a radar, and the inventor makes corresponding exploration in consideration of solving the problems.
Disclosure of Invention
The present application is directed to solving the above problems and providing a method for detecting obstacles in a formula car, a corresponding apparatus, an electronic device, and a computer-readable storage medium.
In order to meet the purposes of the application, the application adopts the following technical scheme:
One of the objects of the present application is to provide a method for detecting an obstacle in a formula car, comprising:
responding to the obstacle detection instruction of the equation motorcycle race to obtain a track field Jing Tuxiang in a monocular camera of the equation motorcycle race;
Performing target detection on the track scene image based on a pre-trained cone detection model, and determining two-dimensional frames of cones of various colors in the track scene image;
Based on a two-dimensional frame of the cone barrel, assuming a three-dimensional frame of the cone barrel, constructing a projection combination related to the three-dimensional frame and the two-dimensional frame, and based on projection point pairs in the projection combination, constructing an overdetermined equation set of related pixel coordinates and three-dimensional coordinates of the cone barrel, solving the overdetermined equation set by adopting a least square method, and determining the three-dimensional position coordinates of the cone barrel in a camera coordinate system;
And determining the distance between the cone barrels of each color type and the formula car based on the three-dimensional position coordinates, and indicating the formula car to adjust the driving direction according to the distance so as to finish the detection of the obstacle of the formula car.
Optionally, the step of training the cone detection model includes:
Acquiring a single training sample and a supervision label thereof in a training set, inputting the training sample into a preset cone detection model, and extracting image characteristic information of an area corresponding to coordinates marked by a corresponding supervision label in a track scene image of the training sample;
the image characteristic information is classified and mapped to preset classification spaces corresponding to the position information and the color types of a plurality of cone barrels, the classification probability corresponding to each classification space is obtained, the position information and the color types of the cone barrels represented by the classification space with the largest classification probability are determined,
Calculating the position information of the cone represented by the classification space with the maximum classification probability and the loss value corresponding to the color type of the cone based on the position information of the cone and the color type of the cone marked by the supervision label by adopting a loss function,
And when the loss values of the various items reach a preset threshold value, indicating that the cone detection model is trained to a convergence state so as to complete training of the cone detection model.
Optionally, the step of constructing a projection combination of the three-dimensional frame and the two-dimensional frame based on the two-dimensional frame of the cone and assuming the three-dimensional frame of the cone, and constructing an overdetermined equation set of the associated pixel coordinates and the three-dimensional coordinates of the cone based on the projection point pairs in the projection combination includes:
taking the central point of the three-dimensional frame of the cone barrel in the three-dimensional space as the origin of a world coordinate system, and determining eight corner points of the three-dimensional frame of the cone barrel under the world coordinate system;
And projecting eight corner points of the three-dimensional frame of the cone under the world coordinate system into a camera coordinate system to construct a projection combination of the three-dimensional frame and the two-dimensional frame, and constructing an overdetermined equation set of the associated pixel coordinate and the three-dimensional coordinate of the cone based on projection point pairs in the projection combination.
Optionally, the step of solving the overdetermined equation set by using a least square method to determine the three-dimensional position coordinates of the cone in the camera coordinate system includes:
Solving the overdetermined equation set according to a plurality of projection combinations of the three-dimensional frame and the two-dimensional frame by adopting a least square method;
And projecting the center of the three-dimensional frame of the cone barrel obtained by each projection combination onto a two-dimensional image to determine a projection point, comparing the projection point with the center point of the two-dimensional frame of the cone barrel, and selecting a solution corresponding to the projection combination with the minimum pixel distance between the two points as the three-dimensional position coordinate of the cone barrel.
Optionally, the cone of each color class includes one or more of a red cone, a blue cone and a yellow cone;
the red cone represents an obstacle on the left side of the unmanned aerial vehicle racing track, the blue cone represents an obstacle on the right side of the unmanned aerial vehicle racing track, and the yellow cone represents a starting point or an ending point of the unmanned aerial vehicle racing track.
Optionally, determining the distance between the cone barrels of each color class and the formula car based on the three-dimensional position coordinates, and indicating the formula car to adjust the driving direction according to the distance, including:
determining three-dimensional position coordinates of the cone barrels of all color types relative to the monocular camera, and determining the distances of the cone barrels of all color types relative to the formula car according to the three-dimensional position coordinates;
and detecting whether the distance is smaller than a preset collision distance, and if so, indicating the formula car to adjust the running direction based on the cone barrels with different colors so as to avoid collision of the formula car.
Optionally, the basic network architecture of the cone detection model is YoloV model.
Another object of the present application is to provide an obstacle detection device for a formula car, comprising:
the track image acquisition module is used for responding to the obstacle detection instruction of the equation motorcycle race and acquiring a track field Jing Tuxiang in a monocular camera of the equation motorcycle race;
The cone barrel two-dimensional frame determining module is used for carrying out target detection on the track scene image based on a pre-trained cone barrel detection model and determining the two-dimensional frames of cone barrels of various colors in the track scene image;
The overdetermined equation set determining module is arranged to construct a projection combination of the three-dimensional frame and the two-dimensional frame based on the assumption of the three-dimensional frame of the cone barrel based on the two-dimensional frame of the cone barrel, and construct an overdetermined equation set of the associated pixel coordinates and the three-dimensional coordinates of the cone barrel based on the projection point pairs in the projection combination;
The three-dimensional coordinate determining module is used for solving the overdetermined equation set by adopting a least square method and determining the three-dimensional position coordinate of the cone barrel in a camera coordinate system;
And the obstacle detection module is used for determining the distance between the cone barrels of each color type and the formula car based on the three-dimensional position coordinates, and indicating the formula car to adjust the driving direction according to the distance so as to finish the detection of the obstacle of the formula car.
An electronic device adapted for another object of the present application comprises a central processor and a memory, said central processor being adapted to invoke the steps of running a computer program stored in said memory for performing the method for detecting obstacles in formula car according to the application.
A computer readable storage medium adapted to another object of the present application stores a computer program implemented according to the method for detecting obstacles in equation (a) for car of the unmanned aerial vehicle in the form of computer readable instructions, which when invoked by a computer, performs the steps included in the corresponding method.
Compared with the prior art, the application aims at the problems that in the prior art, for cone detection on an unmanned equation track, the two-dimensional detection accuracy of the cone is low, the three-dimensional detection accuracy is reduced, the three-dimensional position of the cone is obtained by multi-line laser radar or information fusion of a camera and a radar, the hardware cost is high, and the like, and the application comprises the following beneficial effects:
According to the application, by acquiring monocular camera information and cone priori size information, firstly acquiring image coordinates of a cone two-dimensional frame, then carrying out conversion of a space coordinate system according to the assumption of the two-dimensional frame as a three-dimensional frame, and finally establishing a three-dimensional space coordinate system to solve the cone position. Under the condition of strong priori, the application realizes the monocular three-dimensional target detection of the cone barrel, and has the characteristics of low hardware cost, high accuracy and stable identification;
furthermore, the method for detecting the obstacle of the equation motorcycle race can prevent the equation motorcycle race from impacting the obstacle of the cone in the race track by accurately and rapidly detecting the position and the color type of the obstacle of the cone of each color type, can greatly improve the reactivity of the equation motorcycle race to avoid the obstacle of the cone of each color type, so as to remarkably improve the running speed of the equation motorcycle race, ensure the rapid and orderly running of the equation motorcycle race, and greatly improve the race score of the equation motorcycle race.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is an exemplary network architecture for use with the method of detecting obstructions in formula car of the present application;
FIG. 2 is a diagram of world coordinate system construction and two-dimensional frame pixel coordinates in accordance with an embodiment of the present application;
FIG. 3 is a schematic diagram of a three-dimensional detection frame according to an embodiment of the present application;
FIG. 4 is a schematic view of a three-dimensional position bird's eye view of a cone in an embodiment of the application;
FIG. 5 is a schematic block diagram of an obstacle detection device for formula car according to an embodiment of the application;
Fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, "client," "terminal device," and "terminal device" are understood by those skilled in the art to include both devices that include only wireless signal receivers without transmitting capabilities and devices that include receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include cellular or other communication devices such as Personal computers, tablet computers, cellular or other communication devices having a single-wire or multi-wire display or no multi-wire display, PCS (Personal Communications Service, personal communication system) which may combine voice, data processing, facsimile and/or data communication capabilities, PDA (Personal DIGITAL ASSISTANT ) which may include a radio frequency receiver, pager, internet/intranet access, web browser, notepad, calendar and/or GPS (Global Positioning System ) receiver, conventional laptop and/or palmtop computer or other device having and/or including a radio frequency receiver. As used herein, "client," "terminal device" may be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or adapted and/or configured to operate locally and/or in a distributed fashion, at any other location(s) on earth and/or in space. As used herein, a "client," "terminal device," or "terminal device" may also be a communication terminal, an internet terminal, or a music/video playing terminal, for example, may be a PDA, a MID (Mobile INTERNET DEVICE ), and/or a Mobile phone with a music/video playing function, or may also be a device such as a smart tv, a set top box, or the like.
The application refers to hardware such as a server, a client, a service node, and the like, which essentially is an electronic device with personal computer and other functions, and is a hardware device with necessary components disclosed by von neumann principles such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, an output device, and the like, wherein a computer program is stored in the memory, and the central processing unit calls the program stored in the memory to run, executes instructions in the program, and interacts with the input and output devices, thereby completing specific functions.
It should be noted that the concept of the present application, called "server", is equally applicable to the case of server clusters. The servers should be logically partitioned, physically separate from each other but interface-callable, or integrated into a physical computer or group of computers, according to network deployment principles understood by those skilled in the art. Those skilled in the art will appreciate this variation and should not be construed as limiting the implementation of the network deployment approach of the present application.
One or more technical features of the present application, unless specified in the clear, may be deployed either on a server for implementation and the client remotely invokes an online service interface provided by the acquisition server for implementation of the access, or may be deployed and run directly on the client for implementation of the access.
The neural network model cited or possibly cited in the application can be deployed on a remote server and can be used for implementing remote call on a client, or can be deployed on a client with sufficient equipment capability for direct call, unless specified by plaintext, and in some embodiments, when the neural network model runs on the client, the corresponding intelligence can be obtained through migration learning so as to reduce the requirement on the running resources of the hardware of the client and avoid excessively occupying the running resources of the hardware of the client.
The various data related to the present application, unless specified in the plain text, may be stored either remotely in a server or in a local terminal device, as long as it is suitable for being invoked by the technical solution of the present application.
It will be appreciated by those skilled in the art that the various methods of the application, although described based on the same concepts as one another in common, may be performed independently of one another unless otherwise indicated. Similarly, for the various embodiments disclosed herein, all concepts described herein are presented based on the same general inventive concept, and thus, concepts described herein with respect to the same general inventive concept, and concepts that are merely convenient and appropriately modified, although different, should be interpreted as equivalents.
The various embodiments of the present application to be disclosed herein, unless the plain text indicates a mutually exclusive relationship with each other, the technical features related to the various embodiments may be cross-combined to flexibly construct a new embodiment as long as such combination does not depart from the inventive spirit of the present application and can satisfy the needs in the art or solve the deficiencies in the prior art. This variant will be known to the person skilled in the art.
In recent years, with the development of mathematics, sensor technology and computer hardware technology. In this context, the chinese automobile engineering society and 2017 began to hold chinese college student universities (FSAC). As a key technology in unmanned, the accuracy, the real-time performance and the accuracy of the target detection have important influence on the overall stability of the subsequent unmanned system. The FSAC racing track is mainly formed by enclosing a cone-shaped cone barrel, wherein the left side of the racing track is provided with a red cone barrel, and the right side of the racing track is provided with a blue cone barrel. There are also large yellow cones and small yellow cones at the starting and stopping positions of the racing car.
Referring to fig. 1, referring to the above exemplary scenario, in one embodiment, the method for detecting an obstacle in a formula car of the present application includes:
Step S10, responding to an obstacle detection instruction of the equation motorcycle race to obtain a track field Jing Tuxiang in a monocular camera of the equation motorcycle race;
The terminal device of the equation motorcycle race can respond to the equation motorcycle race obstacle detection instruction to obtain a track field Jing Tuxiang in a monocular camera of the equation motorcycle race, wherein the track scene image comprises a road, various color types of cones and the like, each color type of cone comprises one or more of a red cone, a blue cone and a yellow cone, the red cone represents an obstacle on the left side of the equation motorcycle race, the blue cone represents an obstacle on the right side of the equation motorcycle race, and the yellow cone represents a starting point or an ending point of the equation motorcycle race.
In some embodiments, after the track scene image in the monocular camera of the formula car is obtained, the track scene image can be subjected to data enhancement, noise addition, part deletion, rotation, clipping and translation operations are performed on the track scene image to perform data enhancement, so that the accuracy and the robustness of the cone barrel detection model in target detection on the track scene image are improved, and the data-enhanced image adopts a semi-automatic labeling mode, so that the data labeling efficiency can be greatly improved.
In some embodiments, samples are processed by randomly selecting at least one data enhancement method such as noise adding, part deleting, rotation, cutting and translation operation, ten samples are expanded for each sample in the original data set to obtain 4410 new samples, and finally 4410 marked sample pictures are obtained through semi-automatic marking, so that the semi-automatic tool marking effect is poor. According to the ratio of 7:3, 3528 pieces are randomly divided to serve as training sets and 882 pieces serve as verification sets, after data enhancement is adopted, training data quantity can be increased, generalization capability of a model is improved, noise data is increased, robustness of a two-dimensional detection model is improved, and a more accurate two-dimensional detection frame is provided for three-dimensional target detection.
Step S20, performing target detection on the track scene image based on a pre-trained cone detection model, and determining two-dimensional frames of cones of various colors in the track scene image;
After a track scene image in a monocular camera of the equation motorcycle race is acquired, performing target detection on the track scene image based on a pre-trained cone detection model, and determining two-dimensional frames of cones of various colors such as a red cone, a blue cone and a yellow cone in the track scene image.
Training the cone detection model, comprising the following steps:
Step S201, acquiring a single training sample and a supervision label thereof in a training set, inputting the training sample into a preset cone detection model, and extracting image characteristic information of an area corresponding to coordinates marked by a corresponding supervision label in a track scene image of the training sample;
step 203, mapping the image feature information to preset classification spaces corresponding to the position information and the color types of the plurality of cone barrels in a classification manner, obtaining classification probabilities corresponding to the classification spaces in a mapping manner, and determining the position information and the color types of the cone barrels represented by the classification space with the largest classification probability;
step S205, calculating the position information of the cone represented by the classification space with the maximum classification probability and a loss value corresponding to the color type of the cone based on the position information of the cone and the color type of the cone marked by the supervision label by adopting a loss function;
And S207, when the loss values of the various items reach a preset threshold value, indicating that the cone detection model is trained to a convergence state, so as to complete training of the cone detection model.
Specifically, the basic network architecture of the cone detection model may be YoloV models, etc., the cone detection model may be pre-acquired with the stage scene images of cones with various color types as training samples, the stage scene images may originate from historical video frames or images in unmanned equation events, etc., the stage scene images are labeled with their supervision labels correspondingly for various training samples, the position information of the cones in the stage scene images of the training samples and the color types of the cones are labeled, on the basis of completing the labeling of various training samples, the various training samples and their supervision labels may be mapped to form training sets, the single training sample in the training set and their supervision labels are acquired, the training samples are input into the preset cone detection model, the image feature information of the region corresponding to the coordinates labeled by the corresponding supervision labels in the stage scene images of the training samples is extracted, the image feature information is classified and mapped to the position information of the cone with the preset characteristic, the cone types are mapped to the space corresponding to obtain the probability corresponding to the classification space, the largest classification probability of the cone is determined, the cone position information with the largest classification probability is mapped to the cone types, the cone type is calculated based on the position information of the cone type of the cone, the cone is calculated based on the state information of the cone type is the largest, the color loss is calculated when the color loss is calculated by the cone type model is calculated based on the state information of the cone detection model, and the color loss is calculated by the cone type model is calculated, the model can be subjected to gradient update according to various loss values, the model is further approximately converged by reversely propagating and correcting weight parameters of various links of the model, then iterative training is carried out on the model by continuously calling the next training sample in the training set until the model is trained to a converged state, and it is easy to understand that the cone barrel detection model after training until convergence can detect the position information and the color types of the cone barrel in the current track scene image in a game picture, and the two-dimensional frame of the cone barrel of each color type in the track scene image is determined.
Step S30, a projection combination of the three-dimensional frame and the two-dimensional frame is constructed based on the assumption of the three-dimensional frame of the cone barrel, an overdetermined equation set of associated pixel coordinates and three-dimensional coordinates of the cone barrel is constructed based on projection point pairs in the projection combination, and a least square method is adopted to solve the overdetermined equation set, so that three-dimensional position coordinates of the cone barrel in a camera coordinate system are determined;
After two-dimensional frames of cone barrels of various colors such as a red cone barrel, a blue cone barrel and a yellow cone barrel in the racetrack scene image are determined, assuming the three-dimensional frames of the cone barrels based on the two-dimensional frames of the cone barrels of various colors such as the red cone barrel, the blue cone barrel and the yellow cone barrel, constructing a projection combination related to the three-dimensional frames and the two-dimensional frames, and constructing an overdetermined equation set related to pixel coordinates and cone barrel three-dimensional coordinates based on projection point pairs in the projection combination;
a step of constructing a projection combination of the three-dimensional frame and the two-dimensional frame based on a two-dimensional frame of the cone, and constructing an overdetermined equation set of the associated pixel coordinates and the three-dimensional coordinates of the cone based on projection point pairs in the projection combination, wherein the step comprises the following steps:
Step S301, using the central point of the three-dimensional frame of the cone barrel in the three-dimensional space as the origin of a world coordinate system, and determining eight corner points of the three-dimensional frame of the cone barrel under the world coordinate system;
And step S303, projecting eight corner points of the three-dimensional frame of the cone under the world coordinate system into a camera coordinate system to construct a projection combination of the three-dimensional frame and the two-dimensional frame, and constructing an overdetermined equation set of the associated pixel coordinate and the three-dimensional coordinate of the cone based on projection point pairs in the projection combination.
Further, the step of solving the overdetermined equation set by using a least square method to determine the three-dimensional position coordinates of the cone barrel in a camera coordinate system comprises the following steps:
Step S3001, solving the overdetermined equation set according to a plurality of projection combinations of the three-dimensional frame and the two-dimensional frame by adopting a least square method;
Step S3003, projecting the center of the three-dimensional frame of the cone barrel obtained by each projection combination onto a two-dimensional image to determine a projection point, comparing the projection point with the center point of the two-dimensional frame of the cone barrel, and selecting a solution corresponding to the projection combination with the minimum pixel distance between the two points as the three-dimensional position coordinate of the cone barrel.
Specifically, based on the two-dimensional frames of the red cone, the blue cone, the yellow cone and other cone types, the three-dimensional frames of the cone are assumed, and a priori scene is combined, so that projection combinations of eight angular points and pixel coordinates of 64 three-dimensional frames can be constructed, in order to simplify calculation, the central point of the three-dimensional frames of the cone types of the red cone, the blue cone, the yellow cone and other cone types in the three-dimensional space is taken as the origin of a world coordinate system, meanwhile, the length and width of a target are taken as the directions of three coordinates of the world coordinate system, and the size of the priori target is taken as the length, the width and the height of the three-dimensional frames, wherein dx, dy and dz are the a priori width, the height and the length of the cone respectively, and the eight angular points of the cone types of the red cone, the blue cone, the yellow cone and other cone types in the world coordinate system are:
further, based on the projection point pairs in the projection combination, an overdetermined equation set of the associated pixel coordinates and the three-dimensional coordinates of the cone barrel is constructed, and the specific steps are as follows:
projected from the world coordinate system to the camera coordinate system:
in the formula (1), X c is the X coordinate under the camera coordinate system, X w is the X coordinate under the world coordinate system, and R 3×3、T3×1 is the rotation translation matrix respectively;
From camera coordinate system to pixel coordinate system:
In the formula (2), u and v represent pixel coordinates, dx and dy represent pixel sizes, f represents a focal length of the camera, and the combination of the formula (1) and the formula (2) can be expressed as follows:
K in the formula (3) is a camera internal reference matrix, and can be obtained by calibrating camera internal references.
The deformation can be performed by the formula (3) as follows:
r 3×3 and [ X w Yw Zw]T ] are used as known amounts, and all the known amounts are combined and replaced with a conversion matrix M to obtain the formula (5) shown below.
From equation (5), the coordinate Z c of a point in the Z direction under the camera coordinate system can be expressed as:
Substituting equation 6 into equation (5) above, equations (7) and (8) for pixel coordinates u, v, respectively, can be expressed as follows:
referring to fig. 2,3 and 4, based on the assumption of the three-dimensional frame of the cone, the projection combinations associated with the three-dimensional frame and the two-dimensional frame are constructed, 64 projection combinations can be constructed, each combination contains 4 projection point pairs, the 4 projection point pairs can extract four coordinate values u min,umax,vmin,vmax of the two-dimensional detection frame of the cone, as shown in fig. 2, the four coordinate values u min,umax,vmin,vmax of the two-dimensional detection frame of the cone are respectively substituted into the formula (7) and the formula (8) by pixel coordinate values (u max,vmax) of the upper left corner (u min,vmin) and the lower right corner of the two-dimensional frame, an overdetermined equation set (9) with the number of equations of 4 for unknowns can be constructed, and a translation matrix of T 3×1 (world coordinate system and camera coordinate system) can be obtained by a least square method. Because the world coordinate system is established on the three-dimensional center of the target, the solved T 3×1 is the x, y and z coordinate position of the camera relative to the target. Therefore, -T 3×1 is the three-dimensional position information of the target in the camera coordinate system, and the overdetermined equation set (9) is expressed as follows:
The 64 projection combinations are solved to obtain 64-T 3×1, the center of the target three-dimensional frame obtained by each combination is projected onto a two-dimensional image, the projection points are compared with the center point of the target two-dimensional frame obtained by the cone detection model, and the-T 3×1 corresponding to the projection combination with the minimum pixel distance between the two points is selected as the three-dimensional position coordinate of the target.
And S50, determining the distance between the cone barrels of each color type and the formula car based on the three-dimensional position coordinates, and indicating the formula car to adjust the driving direction according to the distance so as to finish the detection of the obstacle of the formula car.
After the three-dimensional position coordinates of the cone barrels in the camera coordinate system are determined, the distances of the cone barrels of all color types relative to the formula car are determined based on the three-dimensional position coordinates, and the formula car is instructed to adjust the driving direction according to the distances so as to finish detection of the obstacle of the formula car.
In some embodiments, determining the distance of the cone of each color category relative to the formula car based on the three-dimensional position coordinates, and indicating the formula car to adjust the driving direction according to the distance comprises:
Step S501, determining three-dimensional position coordinates of the cone barrels of all color types relative to the monocular camera, and determining the distances of the cone barrels of all color types relative to the formula car according to the three-dimensional position coordinates;
Step S503, detecting whether the distance is smaller than a preset collision distance, if so, indicating the formula car to adjust the driving direction based on the cones with different colors so as to avoid collision of the formula car.
Specifically, after three-dimensional position coordinates of red cone, blue cone, yellow cone and other cone types are determined, the red cone characterizes left obstacle of the equation track, the blue cone characterizes right obstacle of the equation track, the yellow cone characterizes start or end of the equation track, the three-dimensional position coordinates of red cone, blue cone, yellow cone and other cone types are used for determining distance between the cone of each color type and the equation car, detecting whether the distance between the cone of each color type and the equation car is smaller than a preset collision distance, if so, indicating that the equation car adjusts the driving direction based on the equation car of different colors, indicating that the equation car adjusts the driving direction leftwards when the cone is the blue cone, and indicating that the equation car can collide with the equation car rightwards when the cone is the red cone, and ensuring that the equation car can collide with the equation car rightwards when the cone is the red cone.
Compared with the prior art, the method and the device aim at the problems that in the prior art, for cone detection on an unmanned equation track, the two-dimensional detection accuracy of the cone is low, the three-dimensional detection accuracy is reduced, the three-dimensional position of the cone is obtained by multi-line laser radar or information fusion of a camera and a radar, the hardware cost is high, and the like, and the method and the device comprise the following advantages:
According to the application, by acquiring monocular camera information and cone priori size information, firstly acquiring image coordinates of a cone two-dimensional frame, then carrying out conversion of a space coordinate system according to the assumption of the two-dimensional frame as a three-dimensional frame, and finally establishing a three-dimensional space coordinate system to solve the cone position. Under the condition of strong priori, the application realizes the monocular three-dimensional target detection of the cone barrel, and has the characteristics of low hardware cost, high accuracy and stable identification;
furthermore, the method for detecting the obstacle of the equation motorcycle race can prevent the equation motorcycle race from impacting the obstacle of the cone in the race track by accurately and rapidly detecting the position and the color type of the obstacle of the cone of each color type, can greatly improve the reactivity of the equation motorcycle race to avoid the obstacle of the cone of each color type, so as to remarkably improve the running speed of the equation motorcycle race, ensure the rapid and orderly running of the equation motorcycle race, and greatly improve the race score of the equation motorcycle race.
Referring to fig. 5, an apparatus for detecting an obstacle in a formula car according to the present application includes a track image acquisition module 1100, a cone two-dimensional frame determination module 1200, an overdetermined equation set determination module 1300, a three-dimensional coordinate determination module 1400, and an obstacle detection module 1500. The system comprises a track image acquisition module 1100, a cone two-dimensional frame determination module 1200, an overdetermined equation set determination module 1300, a three-dimensional coordinate determination module 1400 and an obstacle detection module 1500, wherein the track image acquisition module 1100 is configured to respond to an obstacle detection instruction of an equation car for the unmanned aerial vehicle to acquire a track field Jing Tuxiang in a monocular camera of the equation car for the unmanned aerial vehicle, the cone two-dimensional frame determination module 1200 is configured to perform target detection on the track scene image based on a pretrained cone detection model to determine two-dimensional frames of cones of various colors in the track scene image, the overdetermined equation set determination module 1300 is configured to construct a projection combination of the three-dimensional frames and the two-dimensional frames based on the two-dimensional frames of the cones to construct an overdetermined equation set of associated pixel coordinates and cone three-dimensional coordinates based on projection point pairs in the projection combination, the three-dimensional coordinate determination module 1400 is configured to solve the overdetermined equation set by adopting a least square method to determine three-dimensional position coordinates of the cones in a camera coordinate system, and the obstacle detection module 1500 is configured to determine distances of cones of various colors relative to the equation cars of the equation car for the unmanned aerial vehicle based on the three-dimensional position coordinates to instruct the equation car to complete the detection of the unmanned aerial vehicle to detect the obstacle.
On the basis of any embodiment of the present application, referring to fig. 6, another embodiment of the present application further provides an electronic device, where the electronic device may be implemented by a computer device, and as shown in fig. 6, the internal structure of the computer device is schematically shown. The computer device includes a processor, a computer readable storage medium, a memory, and a network interface connected by a system bus. The computer readable storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store a control information sequence, and when the computer readable instructions are executed by the processor, the processor can realize the method for detecting the obstacle of the formula car. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, cause the processor to perform the method of the present application for detecting a vehicle in which the equation motorcycle is a car obstacle. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in FIG. 6 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
The processor in this embodiment is configured to execute specific functions of each module and its sub-module in fig. 5, and the memory stores program codes and various data required for executing the above modules or sub-modules. The network interface is used for data transmission between the user terminal or the server. The memory in this embodiment stores program codes and data required for executing all modules/sub-modules in the device for detecting the obstacle in the formula car according to the present application, and the server can call the program codes and data of the server to execute the functions of all the sub-modules.
The present application also provides a storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method for detecting an obstacle in formula car according to any one of the embodiments of the present application.
The present application also provides a computer program product comprising computer programs/instructions which when executed by one or more processors implement the steps of the method for detecting an obstacle in a formula car according to any one of the embodiments of the application.
Those skilled in the art will appreciate that all or part of the processes implementing the methods of the above embodiments of the present application may be implemented by a computer program for instructing relevant hardware, where the computer program may be stored on a computer readable storage medium, where the program, when executed, may include processes implementing the embodiments of the methods described above. The storage medium may be a computer readable storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.
In summary, according to the method for detecting the obstacle of the equation motorcycle race, the positions and the color types of the cone obstacles of each color type are accurately and rapidly detected, so that the equation motorcycle race can be prevented from impacting the cone obstacles in the race track, the reactivity of the equation motorcycle race for avoiding the cone obstacles of each color type can be greatly improved, the running speed of the equation motorcycle race is remarkably improved, the rapid and orderly running of the equation motorcycle race is ensured, and the race score of the equation motorcycle race is greatly improved.

Claims (8)

1.一种无人驾驶方程式赛车障碍物检测方法，其特征在于，包括：1. A method for detecting obstacles in an unmanned formula car, comprising: 响应无人驾驶方程式赛车障碍物检测指令，获取无人驾驶方程式赛车的单目相机中的赛道场景图像；Responding to the obstacle detection command of the unmanned formula car, obtaining a track scene image from the monocular camera of the unmanned formula car; 基于预训练的锥桶检测模型对所述赛道场景图像进行目标检测，确定所述赛道场景图像中各个颜色种类的锥桶的二维框；Performing target detection on the track scene image based on a pre-trained cone detection model to determine the two-dimensional frames of cones of various colors in the track scene image; 基于所述锥桶的二维框假设所述锥桶的三维框，构建所述三维框与二维框相关联的投影组合，基于所述投影组合中的投影点对以构造关联像素坐标和锥桶三维坐标的超定方程组，其包括：Based on the two-dimensional frame of the cone barrel, a three-dimensional frame of the cone barrel is assumed, a projection combination associated with the three-dimensional frame and the two-dimensional frame is constructed, and based on the projection point pairs in the projection combination, an overdetermined equation group associating pixel coordinates and the three-dimensional coordinates of the cone barrel is constructed, which includes: 将三维空间中锥桶的三维框的中心点作为世界坐标系的原点，确定在世界坐标系下所述锥桶的三维框的八个角点；Taking the center point of the three-dimensional frame of the cone barrel in the three-dimensional space as the origin of the world coordinate system, and determining the eight corner points of the three-dimensional frame of the cone barrel in the world coordinate system; 将在世界坐标系下所述锥桶的三维框的八个角点投影至相机坐标系中，以构建所述三维框与二维框的投影组合，基于所述投影组合中的投影点对以构造关联像素坐标和锥桶三维坐标的超定方程组；Projecting the eight corner points of the three-dimensional frame of the cone barrel in the world coordinate system into the camera coordinate system to construct a projection combination of the three-dimensional frame and the two-dimensional frame, and constructing an overdetermined set of equations relating pixel coordinates and the three-dimensional coordinates of the cone barrel based on the projection point pairs in the projection combination; 采用最小二乘法对所述超定方程组进行求解，确定所述锥桶在相机坐标系的三维位置坐标，其包括：The overdetermined set of equations is solved using the least squares method to determine the three-dimensional position coordinates of the cone barrel in the camera coordinate system, which includes: 采用最小二乘法根据所述三维框与二维框的多个投影组合对所述超定方程组进行求解；Solving the overdetermined system of equations based on multiple projection combinations of the three-dimensional box and the two-dimensional box using a least squares method; 将每一种投影组合得到的锥桶三维框中心投影到二维图像上以确定投影点，将所述投影点与锥桶二维框中心点相比较，选取两个点像素距离最小的投影组合对应的解作为锥桶的三维位置坐标；The center of the cone barrel's 3D frame obtained by each projection combination is projected onto the 2D image to determine the projection point, the projection point is compared with the center point of the cone barrel's 2D frame, and the solution corresponding to the projection combination with the smallest pixel distance between the two points is selected as the 3D position coordinate of the cone barrel; 基于所述三维位置坐标确定各个颜色种类的锥桶相对于无人驾驶方程式赛车的距离，根据所述距离指示所述无人驾驶方程式赛车调整行驶方向，以完成无人驾驶方程式赛车障碍物的检测。The distances of the cone barrels of each color relative to the driverless formula racing car are determined based on the three-dimensional position coordinates, and the driverless formula racing car is instructed to adjust its driving direction according to the distances to complete the obstacle detection of the driverless formula racing car. 2.根据权利要求1所述的无人驾驶方程式赛车障碍物检测方法，其特征在于，训练锥桶检测模型的步骤,包括：2. The obstacle detection method for an unmanned formula car according to claim 1, wherein the step of training the cone detection model comprises: 获取训练集中的单个训练样本及其监督标签，将所述训练样本输入至预设的锥桶检测模型，提取出所述训练样本的赛道场景图像中相应的监督标签所标注的坐标对应的区域的图像特征信息；Obtain a single training sample and its supervisory label from the training set, input the training sample into a preset cone detection model, and extract image feature information of the area corresponding to the coordinates marked by the corresponding supervisory label in the track scene image of the training sample; 将所述图像特征信息分类映射至预设的表征多个锥桶的位置信息以及颜色种类相对应的分类空间，获得映射至各个分类空间相对应的分类概率，确定分类概率最大的分类空间表征的所述锥桶的位置信息以及所述锥桶的颜色种类，The image feature information is classified and mapped to a preset classification space corresponding to the position information and color types of multiple cone barrels, and the classification probability corresponding to each classification space is obtained. The position information of the cone barrel and the color type of the cone barrel represented by the classification space with the largest classification probability are determined. 采用损失函数基于所述监督标签所标注的所述锥桶的位置信息以及所述锥桶的颜色种类，计算所述分类概率最大的分类空间表征的所述锥桶的位置信息以及所述锥桶的颜色种类对应的损失值，A loss function is used to calculate the loss value corresponding to the position information of the cone bucket and the color type of the cone bucket represented by the supervision label, which is the classification space with the maximum classification probability. 当各项所述损失值达到预设阈值时，表明锥桶检测模型已被训练至收敛状态，以完成锥桶检测模型的训练。When the loss values of each item reach the preset threshold, it indicates that the cone bucket detection model has been trained to a convergence state, thereby completing the training of the cone bucket detection model. 3.根据权利要求1所述的无人驾驶方程式赛车障碍物检测方法，其特征在于，所述各个颜色种类的锥桶包括红色锥桶、蓝色锥桶以及黄色锥桶的一项或任意多项；3. The obstacle detection method for an unmanned formula car according to claim 1, wherein the cones of various colors include one or more of red cones, blue cones, and yellow cones; 所述红色锥桶表征无人驾驶方程式赛道左边的障碍物，所述蓝色锥桶表征无人驾驶方程式赛道右边的障碍物，所述黄色锥桶表征无人驾驶方程式赛道的起点或终点。The red cone barrel represents the obstacle on the left side of the driverless formula track, the blue cone barrel represents the obstacle on the right side of the driverless formula track, and the yellow cone barrel represents the starting point or end point of the driverless formula track. 4.根据权利要求3所述的无人驾驶方程式赛车障碍物检测方法，其特征在于，基于所述三维位置坐标确定各个颜色种类的锥桶相对于无人驾驶方程式赛车的距离，根据所述距离指示所述无人驾驶方程式赛车调整行驶方向的步骤，包括：4. The obstacle detection method for an unmanned formula car according to claim 3, wherein the steps of determining the distance of the cones of each color relative to the unmanned formula car based on the three-dimensional position coordinates and instructing the unmanned formula car to adjust its direction according to the distances comprise: 确定各个颜色种类的锥桶相对于单目相机的三维位置坐标，根据所述三维位置坐标确定各个颜色种类的锥桶相对于无人驾驶方程式赛车的距离；Determining the three-dimensional position coordinates of the cone barrels of each color relative to the monocular camera, and determining the distance of the cone barrels of each color relative to the driverless formula car based on the three-dimensional position coordinates; 检测所述距离是否小于预设的碰撞距离，若小于，基于不同颜色的锥桶指示所述无人驾驶方程式赛车调整行驶方向，以避免无人驾驶方程式赛车发生碰撞。Detect whether the distance is less than a preset collision distance. If so, instruct the driverless formula car to adjust its driving direction based on cone barrels of different colors to avoid a collision between the driverless formula car and the driverless formula car. 5.根据权利要求1至4任意一项所述的无人驾驶方程式赛车障碍物检测方法，其特征在于，所述锥桶检测模型的基础网络架构为YoloV7模型。5. The obstacle detection method for an unmanned formula racing car according to any one of claims 1 to 4, characterized in that the basic network architecture of the cone barrel detection model is a YoloV7 model. 6.一种无人驾驶方程式赛车障碍物检测装置，其特征在于，包括:6. An obstacle detection device for an unmanned formula racing car, comprising: 赛道图像获取模块，设置为响应无人驾驶方程式赛车障碍物检测指令，获取无人驾驶方程式赛车的单目相机中的赛道场景图像；a track image acquisition module, configured to respond to an obstacle detection instruction of the unmanned formula car and acquire a track scene image from a monocular camera of the unmanned formula car; 锥桶二维框确定模块，设置为基于预训练的锥桶检测模型对所述赛道场景图像进行目标检测，确定所述赛道场景图像中各个颜色种类的锥桶的二维框；a cone barrel two-dimensional frame determination module, configured to perform target detection on the track scene image based on a pre-trained cone barrel detection model, and determine the two-dimensional frames of cone barrels of various colors in the track scene image; 超定方程组确定模块，设置为基于所述锥桶的二维框假设所述锥桶的三维框，构建所述三维框与二维框相关联的投影组合，基于所述投影组合中的投影点对以构造关联像素坐标和锥桶三维坐标的超定方程组，其包括：An overdetermined equation group determination module is configured to assume a three-dimensional frame of the cone barrel based on the two-dimensional frame of the cone barrel, construct a projection combination associated with the three-dimensional frame, and construct an overdetermined equation group associating pixel coordinates and the three-dimensional coordinates of the cone barrel based on the projection point pairs in the projection combination, which includes: 将三维空间中锥桶的三维框的中心点作为世界坐标系的原点，确定在世界坐标系下所述锥桶的三维框的八个角点；Taking the center point of the three-dimensional frame of the cone barrel in the three-dimensional space as the origin of the world coordinate system, and determining the eight corner points of the three-dimensional frame of the cone barrel in the world coordinate system; 将在世界坐标系下所述锥桶的三维框的八个角点投影至相机坐标系中，以构建所述三维框与二维框的投影组合，基于所述投影组合中的投影点对以构造关联像素坐标和锥桶三维坐标的超定方程组；Projecting the eight corner points of the three-dimensional frame of the cone barrel in the world coordinate system into the camera coordinate system to construct a projection combination of the three-dimensional frame and the two-dimensional frame, and constructing an overdetermined set of equations relating pixel coordinates and the three-dimensional coordinates of the cone barrel based on the projection point pairs in the projection combination; 三维坐标确定模块，设置为采用最小二乘法对所述超定方程组进行求解，确定所述锥桶在相机坐标系的三维位置坐标，其包括：The three-dimensional coordinate determination module is configured to solve the overdetermined equations using the least squares method to determine the three-dimensional position coordinates of the cone barrel in the camera coordinate system, and includes: 采用最小二乘法根据所述三维框与二维框的多个投影组合对所述超定方程组进行求解；Solving the overdetermined system of equations based on multiple projection combinations of the three-dimensional box and the two-dimensional box using a least squares method; 将每一种投影组合得到的锥桶三维框中心投影到二维图像上以确定投影点，将所述投影点与锥桶二维框中心点相比较，选取两个点像素距离最小的投影组合对应的解作为锥桶的三维位置坐标；The center of the cone barrel's 3D frame obtained by each projection combination is projected onto the 2D image to determine the projection point, the projection point is compared with the center point of the cone barrel's 2D frame, and the solution corresponding to the projection combination with the smallest pixel distance between the two points is selected as the 3D position coordinate of the cone barrel; 障碍物检测模块，设置为基于所述三维位置坐标确定各个颜色种类的锥桶相对于无人驾驶方程式赛车的距离，根据所述距离指示所述无人驾驶方程式赛车调整行驶方向，以完成无人驾驶方程式赛车障碍物的检测。The obstacle detection module is configured to determine the distance of the cone barrels of each color relative to the unmanned formula racing car based on the three-dimensional position coordinates, and instruct the unmanned formula racing car to adjust its driving direction according to the distance to complete the detection of obstacles of the unmanned formula racing car. 7.一种电子设备，包括中央处理器和存储器，其特征在于，所述中央处理器用于调用运行存储于所述存储器中的计算机程序以执行如权利要求1至5中任意一项所述的方法的步骤。7. An electronic device comprising a central processing unit and a memory, wherein the central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5. 8.一种计算机可读存储介质，其特征在于，其以计算机可读指令的形式存储有依据权利要求1至5中任意一项所述的方法所实现的计算机程序，该计算机程序被计算机调用运行时，执行相应的方法所包括的步骤。8. A computer-readable storage medium, characterized in that it stores a computer program implemented by the method according to any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, it executes the steps included in the corresponding method.
CN202410140039.7A 2024-01-31 2024-01-31 Method, device, equipment and medium for detecting obstacle of formula car Active CN117831004B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410140039.7A CN117831004B (en) 2024-01-31 2024-01-31 Method, device, equipment and medium for detecting obstacle of formula car

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410140039.7A CN117831004B (en) 2024-01-31 2024-01-31 Method, device, equipment and medium for detecting obstacle of formula car

Publications (2)

Publication Number Publication Date
CN117831004A CN117831004A (en) 2024-04-05
CN117831004B true CN117831004B (en) 2025-10-17

Family

ID=90508092

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410140039.7A Active CN117831004B (en) 2024-01-31 2024-01-31 Method, device, equipment and medium for detecting obstacle of formula car

Country Status (1)

Country Link
CN (1) CN117831004B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118692052A (en) * 2024-06-03 2024-09-24 中国第一汽车股份有限公司 Method, device, electronic device and storage medium for identifying vehicle-to-lane cones
CN118447282B (en) * 2024-07-08 2024-10-22 广东工业大学 Unmanned driving formula racing car environment perception method, device, equipment and medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108416773A (en) * 2018-03-08 2018-08-17 凌云光技术集团有限责任公司 A kind of the PCB aperture method of inspection and system of the over-determined systems based on registration point
CN112560769A (en) * 2020-12-25 2021-03-26 北京百度网讯科技有限公司 Method for detecting obstacle, electronic device, road side device and cloud control platform

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112416000B (en) * 2020-11-02 2023-09-15 北京信息科技大学 Environment perception and navigation methods and steering control methods for driverless formula racing cars

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108416773A (en) * 2018-03-08 2018-08-17 凌云光技术集团有限责任公司 A kind of the PCB aperture method of inspection and system of the over-determined systems based on registration point
CN112560769A (en) * 2020-12-25 2021-03-26 北京百度网讯科技有限公司 Method for detecting obstacle, electronic device, road side device and cloud control platform

Also Published As

Publication number Publication date
CN117831004A (en) 2024-04-05

Similar Documents

Publication Publication Date Title
US11763541B2 (en) Target detection method and apparatus, model training method and apparatus, device, and storage medium
CN117831004B (en) Method, device, equipment and medium for detecting obstacle of formula car
CN113762252B (en) Unmanned aerial vehicle intelligent following target determining method, unmanned aerial vehicle and remote controller
CN111931720B (en) Method, apparatus, computer device and storage medium for tracking image feature points
CN111931764A (en) Target detection method, target detection framework and related equipment
KR20210064123A (en) Method and apparatus for recognizing wearing state of safety belt, electronic device, and storage medium
CN114217303B (en) Target positioning and tracking method and device, underwater robot and storage medium
CN113536876B (en) Image recognition method and related device
US11847837B2 (en) Image-based lane detection and ego-lane recognition method and apparatus
US10776941B2 (en) Optimized neural network structure
CN112580558A (en) Infrared image target detection model construction method, detection method, device and system
CN119756400A (en) Autonomous driving vehicle path planning method, device, equipment and medium based on large language model
WO2024093641A1 (en) Multi-modal-fused method and apparatus for recognizing high-definition map element, and device and medium
CN115147818A (en) Playing mobile phone behavior recognition method and device
CN112446845A (en) Map construction method, map construction device, SLAM system, and storage medium
KR20220055072A (en) Method for indoor localization using deep learning
CN111008622B (en) Image object detection method and device and computer readable storage medium
KR102299902B1 (en) Apparatus for providing augmented reality and method therefor
CN114764822B (en) Image processing method, device and electronic equipment
CN118310532A (en) Robot vision positioning navigation method capable of accessing edge node
EP4619937A1 (en) Matching between 2d and 3d for direct localization
CN119942175B (en) Aircraft detection methods, devices and equipment in optical remote sensing images
CN114821479B (en) Image recognition method, device and equipment
US20250251731A1 (en) Range Estimation In Autonomous Maritime Vehicles
WO2025142231A1 (en) Training apparatus, training method, and non-transitory computer-readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant