CN111191561A - Method, apparatus and computer storage medium for re-identification of non-motor vehicles - Google Patents
Method, apparatus and computer storage medium for re-identification of non-motor vehicles Download PDFInfo
- Publication number
- CN111191561A CN111191561A CN201911358617.XA CN201911358617A CN111191561A CN 111191561 A CN111191561 A CN 111191561A CN 201911358617 A CN201911358617 A CN 201911358617A CN 111191561 A CN111191561 A CN 111191561A
- Authority
- CN
- China
- Prior art keywords
- picture
- feature vector
- motor vehicle
- vector
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 239000013598 vector Substances 0.000 claims abstract description 200
- 238000013528 artificial neural network Methods 0.000 claims description 44
- 238000004590 computer program Methods 0.000 claims description 22
- 238000001514 detection method Methods 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 10
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000036544 posture Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000011840 criminal investigation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a method, a device and a computer storage medium for identifying a non-motor vehicle. The method comprises the following steps: determining a non-motor vehicle picture and a driver picture in the picture to be detected; determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture; fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected; and comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result. Therefore, when the non-motor vehicle is re-identified, the characteristic vector of the driver on the non-motor vehicle is considered, so that the obtained characteristic vector of the picture where the non-motor vehicle is located is more accurate, and the obtained re-identification result can be more reliable.
Description
Technical Field
The present invention relates to the field of image processing, and more particularly, to a method, apparatus, and computer storage medium for non-motor vehicle re-identification.
Background
In video structuring applications, re-identification (ReID) of objects with the same Identification (ID) is important. The re-identification (ReID), also called re-identification, can be applied in security, criminal investigation and other related fields, and is mainly used for finding one or several images which are most similar to a target in a group of images. The target therein may be various objects such as a pedestrian, an automobile, etc.
When the target is a non-motor vehicle, when the non-motor vehicle is identified again at present, feature extraction is directly carried out on a detection area of the non-motor vehicle, part of body parts of a driver are generally contained in the detection area, and the detection area of the non-motor vehicle is greatly interfered by clothes, postures and the like of the driver, so that the accuracy of identifying the non-motor vehicle again is low.
Disclosure of Invention
The invention provides a method and a device for identifying the weight of a non-motor vehicle and a computer storage medium, which can improve the accuracy of identifying the weight of the non-motor vehicle.
According to an aspect of the present invention, there is provided a method for re-identification of a non-motor vehicle, the method comprising:
determining a non-motor vehicle picture and a driver picture in the picture to be detected;
determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected;
and comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
In an implementation manner, the fusing the first feature vector and the second feature vector to obtain the feature vector of the picture to be detected includes:
and performing corresponding bit addition on the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
In one implementation, the feature vector of the picture to be detected is represented as fallAnd the ith element f of the feature vectorall iSatisfies the following conditions:
fall i＝fperson i+fcycle i，i＝1,2,...,N，
wherein f iscycle iRepresenting the first feature vector fcycleThe ith element of (1), fperson iRepresenting the second feature vector fpersonN denotes the number of elements of the feature vector.
In an implementation manner, the fusing the first feature vector and the second feature vector to obtain the feature vector of the picture to be detected includes:
and splicing the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
In one implementation, the feature vector of the picture to be detected is represented as fallAnd satisfies the following conditions:
fall＝[fperson；fcycle]or, alternatively, fall＝[fcycle；fperson]，
Wherein f iscycleRepresenting said first feature vector, fpersonRepresenting the second feature vector.
In one implementation, the determining a first feature vector of the picture of the non-motor vehicle and a second feature vector of the picture of the driver includes:
inputting the non-motor vehicle picture into a trained first neural network to obtain the first feature vector;
and inputting the picture of the driver to a trained second neural network to obtain the second feature vector.
In one implementation, the first neural network is trained from a tagged non-motor vehicle picture dataset and the second neural network is trained from a tagged driver picture dataset.
In one implementation, before performing the comparison, the method further includes:
inputting the database picture into a feature extraction network to obtain a picture feature vector in the database,
wherein the feature extraction network comprises a target detection module, the first neural network, the second neural network, and a feature fusion module.
In one implementation, the database picture is selected from surveillance videos, and the database picture is a picture containing non-motor vehicles and drivers.
According to another aspect of the present invention, there is provided an apparatus for heavy identification of a non-motor vehicle, the apparatus comprising:
the detection unit is used for determining a non-motor vehicle picture and a driver picture in the picture to be detected;
the first determining unit is used for determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
the second determining unit is used for fusing the first characteristic vector and the second characteristic vector to obtain a characteristic vector of the picture to be detected;
and the re-identification unit is used for comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
According to a further aspect of the present invention, there is provided an apparatus for re-identification of a non-motor vehicle, comprising a memory, a processor and a computer program stored on the memory and running on the processor, the processor when executing the computer program implementing the steps of the method for re-identification of a non-motor vehicle of the preceding aspect or any implementation.
According to a further aspect of the present invention, a computer storage medium is provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for re-identification of a non-motor vehicle of the preceding aspect or of any of the implementations.
Therefore, when the non-motor vehicle is re-identified, the characteristic vector of the driver on the non-motor vehicle is considered, so that the obtained characteristic vector of the picture where the non-motor vehicle is located is more accurate, and the obtained re-identification result can be more reliable.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail embodiments of the present invention with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings, like reference numbers generally represent like parts or steps.
FIG. 1 is a schematic block diagram of an electronic device of an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a method for re-identification of a non-motor vehicle in accordance with an embodiment of the present invention;
FIG. 3 is another schematic flow chart of a method for re-identification of a non-motor vehicle in accordance with an embodiment of the present invention;
FIG. 4 is yet another schematic flow chart diagram of a method for re-identification of a non-motor vehicle in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of obtaining feature vectors of a picture to be detected according to an embodiment of the present invention;
FIG. 6 is a schematic block diagram of an apparatus for re-identification of a non-motor vehicle in accordance with an embodiment of the present invention;
FIG. 7 is another schematic block diagram of an apparatus for re-identification of a non-motor vehicle in accordance with an embodiment of the present invention;
fig. 8 is still another schematic block diagram of an apparatus for re-identification of a non-motor vehicle according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of embodiments of the invention and not all embodiments of the invention, with the understanding that the invention is not limited to the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the invention described herein without inventive step, shall fall within the scope of protection of the invention.
The embodiment of the present invention can be applied to an electronic device, and fig. 1 is a schematic block diagram of the electronic device according to the embodiment of the present invention. The electronic device 10 shown in FIG. 1 includes one or more processors 102, one or more memory devices 104, an input device 106, an output device 108, an image sensor 110, and one or more non-image sensors 114, which are interconnected by a bus system 112 and/or otherwise. It should be noted that the components and configuration of the electronic device 10 shown in FIG. 1 are exemplary only, and not limiting, and that the electronic device may have other components and configurations as desired.
The processor 102 may include a Central Processing Unit (CPU) 1021 and a Graphics Processing Unit (GPU) 1022 or other forms of Processing units having data Processing capability and/or Instruction execution capability, such as a Field-Programmable gate array (FPGA) or an Advanced Reduced Instruction set Machine (Reduced Instruction set computer) Machine (ARM), and the like, and the processor 102 may control other components in the electronic device 10 to perform desired functions.
The storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory 1041 and/or non-volatile memory 1042. The volatile Memory 1041 may include, for example, a Random Access Memory (RAM), a cache Memory (cache), and/or the like. The non-volatile Memory 1042 may include, for example, a Read-Only Memory (ROM), a hard disk, a flash Memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium and executed by processor 102 to implement various desired functions. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to an outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The image sensor 110 may take images (e.g., photographs, videos, etc.) desired by the user and store the taken images in the storage device 104 for use by other components.
It should be noted that the components and structure of the electronic device 10 shown in fig. 1 are merely exemplary, and although the electronic device 10 shown in fig. 1 includes a plurality of different devices, some of the devices may not be necessary, some of the devices may be more numerous, and the like, as desired, and the invention is not limited thereto.
For the existing non-motor vehicle weight recognition technology, firstly, a detection model is used to detect the position of a non-motor vehicle, then, pictures of a detection area are sent to a CNN network, and the feature vector of the non-motor vehicle is extracted; and comparing the feature vector of the non-motor vehicle to be retrieved with the feature vector of the non-motor vehicle picture in the retrieval library, selecting other vectors most similar to the non-motor vehicle vector to be retrieved, and sequencing according to the similarity, thereby obtaining a non-motor vehicle retrieval result. However, because the same non-motor vehicle picture has different shooting angles, illumination conditions, driving postures and the like, the uncertain factors increase the difficulty of the CNN network in extracting picture features, so that the features extracted by the network lack generalization capability, further influence the subsequent retrieval sequencing result and reduce the retrieval accuracy.
An embodiment of the present invention provides a method for re-identification of a non-motor vehicle, a schematic flow chart of which may be shown in fig. 2. The method illustrated in FIG. 2 may be performed by the electronic device 10 illustrated in FIG. 1, and in particular by the processor 102. The method shown in fig. 2 may include:
and S10, determining the feature vector of the picture to be detected, wherein the picture to be detected comprises the non-motor vehicle and the driver.
And S20, comparing the feature vector of the picture to be detected with the feature vector of the picture in the database to obtain a re-identification result.
For example, in S10, the picture to be detected may be input to the trained feature extraction network, so as to obtain the feature vector of the picture to be detected. As shown in fig. 3.
Illustratively, pictures in the database may be similarly input to the trained feature extraction network, thereby obtaining feature vectors of the pictures in the database.
That is to say, the same feature extraction network can be used for extracting the feature vector of the picture to be detected and extracting the feature vector of the picture in the database, so that the consistency can be ensured, errors caused by differences among different networks can be avoided, and the accuracy of re-identification can be improved.
As one implementation mode, the feature extraction network can comprise a first neural network, a second neural network and a feature fusion module, wherein the first neural network is used for determining the feature vector of the non-motor vehicle, the second neural network is used for determining the feature vector of the driver, and the feature fusion module is used for obtaining the feature vector of the picture.
Fig. 4 shows another schematic flow chart of the method for recognizing a non-motor vehicle again according to the exemplary embodiment of the present invention. The method shown in fig. 4 includes:
s110, determining a non-motor vehicle picture and a driver picture in the pictures to be detected;
s120, determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
s130, fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected;
and S140, comparing the characteristic vector with the image characteristic vector in the database to obtain a re-identification result.
It should be understood that the drivers referred to in S10 and S110 are drivers of non-motor vehicles. For example, in the picture to be detected, a partial body area of the driver may block a partial area of the non-motor vehicle.
In one embodiment, S10 includes S110, S120, and S130.
Illustratively, S110 may include: the picture to be detected is input to a target detection module (or called target detection model), so as to obtain a non-motor vehicle picture and a driver picture, as shown in fig. 5. Specifically, a non-motor vehicle picture and a driver picture in the pictures to be detected are obtained.
Specifically, the target detection module can detect the position of the non-motor vehicle in the picture to be detected and the position of the driver in the picture to be detected, and further can obtain a picture of the non-motor vehicle and a picture of the driver.
Exemplarily, S120 may include: inputting the non-motor vehicle picture into a trained first neural network to obtain a first feature vector; and inputting the picture of the driver into the trained second neural network to obtain a second feature vector, as shown in fig. 5. Wherein the first neural network and the second neural network may be two independent convolutional neural networks for extracting feature vectors of the non-motor vehicle and the driver, respectively.
The first neural network may be trained from a tagged non-motor vehicle picture dataset and the second neural network may be trained from a tagged driver picture dataset.
The training process for the first neural network and the second neural network may be performed sequentially, which results in relatively low processor requirements. The training process of the first neural network and the second neural network can also be performed in parallel, so that the training efficiency can be improved. Alternatively, the training process for the first neural network and the training process for the second neural network partially overlap in time, which is not limited by the present invention.
Alternatively, a labeled non-motor vehicle picture dataset may be acquired and the first neural network derived by training. Wherein the tag is used to represent the ID of the corresponding non-motor vehicle. And during training, the distance between the feature vectors obtained by different non-motor vehicle pictures with the same label is smaller than the error.
Alternatively, a labeled driver picture data set may be obtained and trained to obtain a second neural network. Wherein the tag is used to represent the ID of the corresponding driver. And during training, the distance between the feature vectors obtained by different driver pictures with the same label is smaller than the error.
As an example, S130 may include: and performing corresponding bit addition on the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
The feature vector of the picture to be detected can be represented as fallExpressing the first feature vector as fcycleExpressing the second feature vector as fperson。
Then, the corresponding bit addition can be expressed as: f. ofall i＝fperson i+fcycle i1, 2. Wherein f isall iFeature vector f representing picture to be detectedallThe ith element of (1), fcycle iRepresenting a first feature vector fcycleThe ith element of (1), fperson iRepresenting a second feature vector fpersonN denotes the number of elements included in the feature vector. Alternatively, other addition methods may be used, such as weighted summation of corresponding bits, and the weight of the element of the first eigenvector may not be equal to (e.g., greater than) the weight of the element of the second eigenvector.
As another example, S130 may include: and splicing the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
The feature vector of the picture to be detected can be represented as fallExpressing the first feature vector as fcycleExpressing the second feature vector as fperson。
Then, the splice can be expressed as: f. ofall＝[fperson；fcycle]Or, alternatively, fall＝[fcycle；fperson]. Alternatively,other splicing methods may be used, such as alternating elements of the first eigenvector and the second eigenvector, etc.
Illustratively, the feature vectors of the pictures in the database may be similarly derived using a target detection module, a first neural network, a second neural network, and a feature fusion module. That is to say, the feature vector of the picture to be detected and the feature vector of the picture in the database can be obtained in the same way, so that consistency can be ensured, errors caused by differences of different ways can be avoided, and accuracy of re-identification can be improved.
In the embodiment of the present invention, a feature extraction network may also be trained in advance, and the feature extraction network is used to obtain the feature vector in S10. The feature extraction network may include a target detection module, a first neural network, a second neural network, and a feature fusion module.
Illustratively, the tagged non-motor vehicle picture dataset and the tagged driver picture dataset may be used for training. For example, the first neural network and the second neural network may be trained by using the contents of the above-mentioned part described in connection with S120, and when feature fusion is performed by using the feature fusion module, fine tuning is performed on the first neural network and the second neural network, so as to obtain the feature extraction network.
For example, the database picture in S20 or S140 may be selected from surveillance videos, and the database picture is a picture including non-motor vehicles and drivers.
Some of the frames of the surveillance video may include only vehicles or pedestrians, but not non-vehicles, and these frames are not selected into the database. That is, the pictures in the database constructed by the embodiment of the present invention all include non-motor vehicles and drivers.
Illustratively, S20 or S140 may include: determining a characteristic vector of each image in a plurality of images in a database to obtain a plurality of database characteristic vectors; and comparing the characteristic vector of the picture to be detected with the characteristic vectors of the databases to obtain a re-identification result.
For example, the feature vector of the picture to be detected and the feature vectors of the databases may be input to the feature comparison module, as shown in fig. 5, to obtain the re-recognition result.
Specifically, the comparison module may calculate a distance between the feature vector of the picture to be detected and each of the plurality of database feature vectors, output a plurality of minimum distances among the obtained plurality of distances, and output a plurality of images in the database corresponding to the plurality of distances as the re-recognition result.
Wherein the distance may be any one of the following distances between two feature vectors: euclidean Distance (Euclidean Distance), Mahalanobis Distance (Mahalanobis Distance), manhattan Distance (Euclidean Distance), Chebyshev Distance (Chebyshev Distance), minkowski Distance (MinkowskiDistance), Hamming Distance (Hamming Distance), Jacgard Distance (Jaccard Distance), and the like.
Wherein, the distances can be sorted from small to large, so as to find the smallest ones. Several of these may be 1, 10, 100, etc.
Therefore, when the non-motor vehicle is re-identified, the characteristic vector of the driver on the non-motor vehicle is considered, so that the obtained characteristic vector of the picture where the non-motor vehicle is located is more accurate, and the obtained re-identification result can be more reliable.
It should be understood that embodiments of the present invention may be applied to a variety of scenarios other than vehicle weight identification. For example, in the security field, when a certain non-motor vehicle is lost, the method of the embodiment of the invention can be adopted to retrieve the possible position of the lost non-motor vehicle from the monitoring video. For example, in the field of intelligent transportation, the method of the embodiment of the invention can be used for obtaining the positions of specific non-motor vehicles at different moments based on the monitoring video, determining the moving tracks of the non-motor vehicles and the like.
Fig. 6 is a schematic block diagram of an apparatus for re-identification of a non-motor vehicle according to an embodiment of the present invention. The apparatus 20 shown in fig. 6 comprises: a determination unit 21 and a re-identification unit 22.
The determining unit 21 may be configured to determine a feature vector of a picture to be detected, where the picture to be detected includes a non-motor vehicle and a driver. The re-recognition unit 22 may be configured to compare the feature vector of the picture to be detected with the feature vectors of the pictures in the database to obtain a re-recognition result.
Exemplarily, as shown in fig. 7, the determining unit 21 may include a detecting unit 210, a first determining unit 220, and a second determining unit 230.
The detection unit 210 is configured to determine a non-motor vehicle picture and a driver picture in the pictures to be detected;
a first determining unit 220, configured to determine a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
a second determining unit 230, configured to fuse the first feature vector and the second feature vector to obtain a feature vector of the to-be-detected picture;
and the re-identification unit 22 is configured to obtain a re-identification result by comparing the feature vector with the picture feature vector in the database.
For example, the detection unit 210 may be referred to as an object detection module, and is used for detecting a non-motor vehicle and a driver in a picture to be detected.
Exemplarily, the second determining unit 230 may be referred to as a feature fusion module, and is configured to fuse the first feature vector and the second feature vector into a feature vector of the picture to be detected.
In an embodiment, the second determining unit 230 may specifically be configured to: and performing corresponding bit addition on the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
For example, the feature vector of the picture to be detected is represented as fallAnd the ith element f of the feature vectorall iSatisfies the following conditions: f. ofall i＝fperson i+fcycl i e1, 2. Wherein f iscycle iRepresenting the first feature vector fcycleThe ith element of (1), fperson iRepresenting the second feature vector fpersonN denotes the number of elements of the feature vector.
In another embodiment, the second determining unit 230 may be specifically configured to: and splicing the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
Wherein the feature vector of the picture to be detected is represented as fallAnd satisfies the following conditions: f. ofall＝[fperson；fcycle]Or, alternatively, fall＝[fcycle；fperson]. Wherein f iscycleRepresenting said first feature vector, fpersonRepresenting the second feature vector.
Exemplarily, the first determining unit 220 may be specifically configured to: inputting the non-motor vehicle picture into a trained first neural network to obtain the first feature vector; and inputting the picture of the driver to a trained second neural network to obtain the second feature vector.
Wherein the first neural network is trained from a tagged non-motor vehicle picture dataset and the second neural network is trained from a tagged driver picture dataset.
Exemplarily, the determining unit 21 may be further configured to input the database picture into a feature extraction network, so as to obtain a picture feature vector in the database. Wherein the feature extraction network comprises a target detection module, the first neural network, the second neural network, and a feature fusion module.
Illustratively, the database picture is selected from surveillance videos, and the database picture is a picture containing non-motor vehicles and drivers.
The device 20 shown in fig. 6 or fig. 7 can implement the method for re-identification of a non-motor vehicle shown in fig. 2 to fig. 5, and is not described herein again to avoid repetition.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In addition, another device for re-identification of a non-motor vehicle is provided in an embodiment of the present invention, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the processor implements the steps of the method for re-identification of a non-motor vehicle shown in fig. 2 to 5.
As shown in fig. 8, the apparatus 30 may include a memory 810 and a processor 820.
The memory 810 stores computer program code for implementing corresponding steps in a method for re-identification of a non-motor vehicle according to an embodiment of the present invention.
The processor 820 is configured to execute the computer program code stored in the memory 810 to perform the corresponding steps of the method for re-identification of a non-motor vehicle according to an embodiment of the present invention.
Illustratively, the computer program code when executed by the processor 820 performs the steps of: determining a characteristic vector of a picture to be detected, wherein the picture to be detected comprises a non-motor vehicle and a driver; and comparing the characteristic vector of the picture to be detected with the characteristic vector of the picture in the database to obtain a re-identification result.
Or, illustratively, when the computer program code is executed by the processor 820, the following steps are performed: determining a non-motor vehicle picture and a driver picture in the picture to be detected; determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture; fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected; and comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
In addition, an embodiment of the present invention further provides an electronic device, which may be the electronic device 10 shown in fig. 1, or may include the apparatus shown in fig. 6, fig. 7, or fig. 8. The electronic device may implement the method for re-identification of a non-motor vehicle as illustrated in fig. 2 to 5 described above.
In addition, the embodiment of the invention also provides a computer storage medium, and the computer storage medium is stored with the computer program. The computer program, when executed by a processor, may implement the steps of the method for re-identification of a non-motor vehicle as described above with reference to fig. 2 to 5. For example, the computer storage medium is a computer-readable storage medium.
In one embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the steps of: determining a characteristic vector of a picture to be detected, wherein the picture to be detected comprises a non-motor vehicle and a driver; and comparing the characteristic vector of the picture to be detected with the characteristic vector of the picture in the database to obtain a re-identification result.
In another embodiment, the computer program instructions, when executed by a computer or processor, cause the computer or processor to perform the steps of: determining a non-motor vehicle picture and a driver picture in the picture to be detected; determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture; fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected; and comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
The computer storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a portable compact disc read only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.
In addition, the embodiment of the present invention also provides a computer program product, which contains instructions that, when executed by a computer, cause the computer to execute the steps of the method for identifying a non-motor vehicle shown in any one of fig. 2 to 5.
Therefore, when the non-motor vehicle is re-identified, the characteristic vector of the driver on the non-motor vehicle is considered, so that the obtained characteristic vector of the picture where the non-motor vehicle is located is more accurate, and the obtained re-identification result can be more reliable.
Although the illustrative embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the foregoing illustrative embodiments are merely exemplary and are not intended to limit the scope of the invention thereto. Various changes and modifications may be effected therein by one of ordinary skill in the pertinent art without departing from the scope or spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as set forth in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the method of the present invention should not be construed to reflect the intent: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
It will be understood by those skilled in the art that all of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where such features are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some of the modules in an item analysis apparatus according to embodiments of the present invention. The present invention may also be embodied as apparatus programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
The above description is only for the specific embodiment of the present invention or the description thereof, and the protection scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (12)
1. A method for re-identification of a non-motor vehicle, the method comprising:
determining a non-motor vehicle picture and a driver picture in the picture to be detected;
determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
fusing the first feature vector and the second feature vector to obtain a feature vector of the picture to be detected;
and comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
2. The method according to claim 1, wherein the fusing the first feature vector and the second feature vector to obtain the feature vector of the picture to be detected comprises:
and performing corresponding bit addition on the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
3. Method according to claim 2, characterized in that the feature vector of the picture to be detected is represented by fallAnd the ith element f of the feature vectorall iSatisfies the following conditions:
fall i＝fperson i+fcycle i，i＝1,2,...,N，
wherein f iscycle iRepresenting the first feature vector fcycleThe ith element of (1), fperson iRepresenting the second feature vector fpersonN denotes the number of elements of the feature vector.
4. The method according to claim 1, wherein the fusing the first feature vector and the second feature vector to obtain the feature vector of the picture to be detected comprises:
and splicing the first characteristic vector and the second characteristic vector to obtain the characteristic vector of the picture to be detected.
5. Method according to claim 4, characterized in that the feature vector of the picture to be detected is represented as fallAnd satisfies the following conditions:
fall＝[fperson；fcycle]or, alternatively, fall＝[fcycle；fperson]，
Wherein f iscycleRepresenting said first feature vector, fpersonRepresenting the second feature vector.
6. The method according to any one of claims 1 to 5, wherein the determining a first feature vector of the picture of non-motor vehicles and a second feature vector of the picture of drivers comprises:
inputting the non-motor vehicle picture into a trained first neural network to obtain the first feature vector;
and inputting the picture of the driver to a trained second neural network to obtain the second feature vector.
7. The method of claim 6, wherein the first neural network is trained from a tagged non-motor vehicle picture dataset and the second neural network is trained from a tagged driver picture dataset.
8. The method of claim 6, further comprising, prior to performing the alignment:
inputting the database picture into a feature extraction network to obtain a picture feature vector in the database,
wherein the feature extraction network comprises a target detection module, the first neural network, the second neural network, and a feature fusion module.
9. The method of claim 8, wherein the database picture is selected from surveillance video, and the database picture is a picture including non-motor vehicles and drivers.
10. A device for re-identification of a non-motor vehicle, characterized in that it is adapted to implement the steps of the method according to any one of claims 1 to 9, said device comprising:
the detection unit is used for determining a non-motor vehicle picture and a driver picture in the picture to be detected;
the first determining unit is used for determining a first feature vector of the non-motor vehicle picture and a second feature vector of the driver picture;
the second determining unit is used for fusing the first characteristic vector and the second characteristic vector to obtain a characteristic vector of the picture to be detected;
and the re-identification unit is used for comparing the characteristic vector with the picture characteristic vector in the database to obtain a re-identification result.
11. An apparatus for re-identification of a non-motor vehicle, comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the steps of the method of any one of claims 1 to 9 when executing the computer program.
12. A computer storage medium on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911358617.XA CN111191561A (en) | 2019-12-25 | 2019-12-25 | Method, apparatus and computer storage medium for re-identification of non-motor vehicles |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911358617.XA CN111191561A (en) | 2019-12-25 | 2019-12-25 | Method, apparatus and computer storage medium for re-identification of non-motor vehicles |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111191561A true CN111191561A (en) | 2020-05-22 |
Family
ID=70710481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911358617.XA Pending CN111191561A (en) | 2019-12-25 | 2019-12-25 | Method, apparatus and computer storage medium for re-identification of non-motor vehicles |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111191561A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343888A (en) * | 2021-06-23 | 2021-09-03 | 苏州科达科技股份有限公司 | Non-motor vehicle identification method, system and device |
| CN114445787A (en) * | 2021-12-31 | 2022-05-06 | 深圳云天励飞技术股份有限公司 | Non-motor vehicle weight recognition method and related equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1495658A (en) * | 2002-06-30 | 2004-05-12 | 贺贵明 | Driver's face image identification and alarm device and method |
| CN107657249A (en) * | 2017-10-26 | 2018-02-02 | 珠海习悦信息技术有限公司 | Method, apparatus, storage medium and the processor that Analysis On Multi-scale Features pedestrian identifies again |
| CN107757625A (en) * | 2016-08-17 | 2018-03-06 | 罗伯特·博世有限公司 | Mirror method for distinguishing is carried out for being changed in a motor vehicle to driver |
| CN107944017A (en) * | 2017-12-11 | 2018-04-20 | 浙江捷尚视觉科技股份有限公司 | The search method of non-motor vehicle in a kind of video |
| CN109214315A (en) * | 2018-08-21 | 2019-01-15 | 北京深瞐科技有限公司 | Across the camera tracking method and device of people's vehicle |
| CN110008379A (en) * | 2019-03-19 | 2019-07-12 | 北京旷视科技有限公司 | Monitoring image processing method and processing device |
-
2019
- 2019-12-25 CN CN201911358617.XA patent/CN111191561A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1495658A (en) * | 2002-06-30 | 2004-05-12 | 贺贵明 | Driver's face image identification and alarm device and method |
| CN107757625A (en) * | 2016-08-17 | 2018-03-06 | 罗伯特·博世有限公司 | Mirror method for distinguishing is carried out for being changed in a motor vehicle to driver |
| CN107657249A (en) * | 2017-10-26 | 2018-02-02 | 珠海习悦信息技术有限公司 | Method, apparatus, storage medium and the processor that Analysis On Multi-scale Features pedestrian identifies again |
| CN107944017A (en) * | 2017-12-11 | 2018-04-20 | 浙江捷尚视觉科技股份有限公司 | The search method of non-motor vehicle in a kind of video |
| CN109214315A (en) * | 2018-08-21 | 2019-01-15 | 北京深瞐科技有限公司 | Across the camera tracking method and device of people's vehicle |
| CN110008379A (en) * | 2019-03-19 | 2019-07-12 | 北京旷视科技有限公司 | Monitoring image processing method and processing device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343888A (en) * | 2021-06-23 | 2021-09-03 | 苏州科达科技股份有限公司 | Non-motor vehicle identification method, system and device |
| CN114445787A (en) * | 2021-12-31 | 2022-05-06 | 深圳云天励飞技术股份有限公司 | Non-motor vehicle weight recognition method and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109961009B (en) | Pedestrian detection method, system, device and storage medium based on deep learning | |
| US9824459B2 (en) | Tracking objects between images | |
| US10990857B2 (en) | Object detection and learning method and apparatus | |
| CN109145680B (en) | Method, device and equipment for acquiring obstacle information and computer storage medium | |
| CN106650662B (en) | Target object shielding detection method and device | |
| CN111931929A (en) | Training method and device of multi-task model and storage medium | |
| JP2004318879A5 (en) | ||
| CN112633255B (en) | Target detection method, device and equipment | |
| CN112541394A (en) | Black eye and rhinitis identification method, system and computer medium | |
| CN114168768A (en) | Image retrieval method and related equipment | |
| CN112926462B (en) | Training method and device, action recognition method and device and electronic equipment | |
| CN115546831A (en) | Cross-modal pedestrian searching method and system based on multi-granularity attention mechanism | |
| CN111444850A (en) | Picture detection method and related device | |
| CN110059542B (en) | Method and related equipment for face liveness detection based on improved Resnet | |
| CN116129523A (en) | Action recognition method, device, terminal and computer readable storage medium | |
| Kumar | SEAT-YOLO: A squeeze-excite and spatial attentive you only look once architecture for shadow detection | |
| CN111680680B (en) | Target code positioning method and device, electronic equipment and storage medium | |
| CN111191561A (en) | Method, apparatus and computer storage medium for re-identification of non-motor vehicles | |
| CN110490058B (en) | Training method, device and system of pedestrian detection model and computer readable medium | |
| CN115115825B (en) | Method, device, computer equipment and storage medium for detecting object in image | |
| Bilgin et al. | Road sign recognition system on Raspberry Pi | |
| CN111444803A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110717406B (en) | Face detection method and device and terminal equipment | |
| CN117423163A (en) | Model training method, smoking behavior detection method and model training device | |
| CN112949526B (en) | Face detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200522 |
|
| RJ01 | Rejection of invention patent application after publication |