CN112990069B - Abnormal driving behavior detection method, device, terminal and medium - Google Patents
Abnormal driving behavior detection method, device, terminal and medium Download PDFInfo
- Publication number
- CN112990069B CN112990069B CN202110349516.7A CN202110349516A CN112990069B CN 112990069 B CN112990069 B CN 112990069B CN 202110349516 A CN202110349516 A CN 202110349516A CN 112990069 B CN112990069 B CN 112990069B
- Authority
- CN
- China
- Prior art keywords
- hand
- human body
- key points
- key point
- ear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 101
- 238000001514 detection method Methods 0.000 title claims abstract description 52
- 230000006399 behavior Effects 0.000 claims abstract description 102
- 238000000034 method Methods 0.000 claims abstract description 20
- 230000001815 facial effect Effects 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 9
- 230000000391 smoking effect Effects 0.000 description 9
- 235000019504 cigarettes Nutrition 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 5
- 239000000779 smoke Substances 0.000 description 5
- 206010000117 Abnormal behaviour Diseases 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 210000000988 bone and bone Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000003571 electronic cigarette Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 235000011888 snacks Nutrition 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, a device, a terminal and a medium for detecting abnormal driving behavior, which are characterized in that the method identifies the human body key points of a driver in a driver image according to a human body key point model by acquiring the driver image, and the position information of the human body key points, if the identified human body key point set comprises target human body key points, acquiring the position relation among the target human body key points, and detecting abnormal driving behaviors. The problem that the abnormal driving behavior is difficult to detect when the driving tool only has one driver is avoided, the detection speed and accuracy of the abnormal driving behavior are improved, and the driving safety can be effectively improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a terminal, and a medium for detecting abnormal driving behavior.
Background
With the development of economy and society, the quantity of the motor vehicles is more and more large, the motor vehicles bring great convenience to travel and transportation of people conveniently and rapidly, but certain traffic risks exist. Generally speaking, a more focused driving motor vehicle can effectively reduce traffic risk, but because a cab is relatively private, many drivers in actual life can perform abnormal driving behaviors such as smoking, watching television, watching novice, answering calls and the like in the driving process of the vehicle, so that certain risk hidden danger is caused.
In the related art, the occurrence or duration of the abnormal driving behavior is often found by means of self-consciousness of a driver, observation of passengers in the same vehicle, and the like, and the reliability is limited, so that a method capable of automatically detecting the abnormal driving behavior is needed to accurately detect the abnormal driving behavior, and driving safety is improved.
Disclosure of Invention
In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a method, apparatus, terminal and medium for detecting abnormal driving behavior, which are used for solving the technical problems of occurrence or duration of abnormal driving behavior and limited reliability found by means of self-consciousness of a driver and observation of passengers in the same vehicle.
In view of the foregoing, the present invention provides an abnormal driving behavior detection method, including:
acquiring an image of a driver;
Identifying human body key points of a driver in the driver image and position information of the human body key points based on a human body key point model;
If the identified human body key point set comprises target human body key points, acquiring the position relation among the target human body key points, and detecting abnormal driving behaviors, wherein the target human body key points comprise human body key points of at least two preset parts.
Optionally, the target human body key points include a hand key point and an ear key point, and the acquiring the positional relationship between the target human body key points includes:
determining a hand-ear distance between the hand key point and the ear key point according to the position information of the hand key point and the position information of the ear key point;
If the hand-ear distance is smaller than the preset hand-ear distance, selecting a hand-ear partial image from the driver image, wherein the hand-ear partial image comprises images of hand key points and positions of ear key points;
And carrying out image recognition on the hand-ear partial images, and if the mobile phone is recognized, determining that abnormal driving behaviors exist.
Optionally, the target human body key points include a hand key point and a lip key point, and the acquiring the positional relationship between the target human body key points includes:
determining a hand-lip distance between a hand key point and a lip key point according to the position information of the hand key point and the position information of the lip key point;
if the hand-lip distance is smaller than the preset hand-lip distance, selecting a hand-lip partial image from the driver image, wherein the hand-lip partial image comprises images of hand key points and positions of lip key points;
and carrying out image recognition on the partial images of the hand lips, and if the slender object is recognized, determining that abnormal driving behaviors exist.
Optionally, determining that abnormal driving behavior exists includes:
Respectively obtaining a first pixel average value of one end of the slender object far away from the lip and a second pixel average value of the partial image of the hand lip;
If the first pixel average value is larger than the second pixel average value, determining that abnormal driving behaviors exist.
Optionally, the method further comprises:
Acquiring position information of a current human body key point of a face, and selecting a face partial image;
identifying facial key points in the facial partial image and position information of the facial key points based on a facial key point model;
Determining a deflection angle according to the position information of the facial key points, wherein the deflection angle comprises at least one of a pitch angle, a roll angle and a yaw angle;
and if the deflection angle is larger than a preset deflection angle threshold value, determining that abnormal driving behaviors exist.
Optionally, the yaw angle is obtained by:
θx＝atan2(-ryz,rzz)；
the roll angle is obtained by:
the pitch angle is obtained by:
θz＝atan2(-rxy,rxx)；
wherein, θ x is yaw angle, θ y is roll angle, and θ z is pitch angle.
Optionally, the training mode of the human body key point model includes:
Obtaining M human body key points in a human body key point image, wherein each human body key point is converted into a maximum activation point in an N matrix through Gaussian distribution;
acquiring a backbone network, inputting the human body key point image into the backbone network, and generating an output result;
and determining L2 loss as a loss function according to the maximum activation point and the output result, and training the backbone network.
The invention also provides an abnormal driving behavior detection device, which comprises:
The acquisition module is used for acquiring the image of the driver;
The identification module is used for identifying human body key points of a driver in the driver image and position information of the human body key points based on a human body key point model;
the detection module is used for acquiring the position relation between the target human body key points if the identified human body key point set comprises the target human body key points, and detecting abnormal driving behaviors, wherein the target human body key points comprise human body key points of at least two preset parts.
The invention also provides a terminal, which comprises a processor, a memory and a communication bus;
the communication bus is used for connecting the processor and the memory;
The processor is configured to execute a computer program stored in the memory to implement the abnormal driving behavior detection method according to any one of the embodiments described above.
The present invention also provides a computer-readable storage medium, having stored thereon a computer program,
The computer program is for causing the computer to execute the abnormal driving behavior detection method according to any one of the embodiments described above.
As described above, the abnormal driving behavior detection method, device, terminal and medium provided by the invention have the following beneficial effects:
According to the method, a driver image is obtained, human body key points of a driver in the driver image and position information of the human body key points are identified according to a human body key point model, and if the identified human body key point set comprises target human body key points, the position relation among the target human body key points is obtained, so that abnormal driving behavior detection is carried out. The problem that the abnormal driving behavior is difficult to detect when the driving tool only has one driver is avoided, the detection speed and accuracy of the abnormal driving behavior are improved, and the driving safety can be effectively improved.
Drawings
Fig. 1 is a schematic flow chart of a method for detecting abnormal driving behavior according to an embodiment of the present invention;
FIG. 2 is a schematic illustration of a human body key point determined from human body skeletal joints;
FIG. 3 is a schematic illustration of a human keypoint image;
FIG. 4 is a schematic illustration of a partial image of a hand and ear;
FIG. 5 is a schematic illustration of a partial image of the lips;
FIG. 6 is a schematic diagram of recognition results of face keypoints recognition on a face partial image;
fig. 7 is a schematic structural diagram of an abnormal driving behavior detection device according to a second embodiment of the present invention;
Fig. 8 is a schematic structural diagram of a terminal according to a second embodiment of the present invention.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.
It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.
Example 1
Referring to fig. 1, an abnormal driving behavior detection method provided by an embodiment of the present invention includes:
s101: an image of the driver is acquired.
Crews include, but are not limited to, crews of steering tools such as motor vehicles, non-motor vehicles, boats, aircraft, etc.
The driver's image may be captured by a capturing device (e.g., an in-vehicle monitoring device installed in the cab, etc.) provided on the driving tool, or may be captured by an outdoor capturing device such as a road monitoring camera, a crossing capturing device, etc.
When the driver image is a video frame extracted based on the driver video, a video frame with good light can be selected as the driver image within a certain time range.
The driver image can be acquired based on external instructions or automatically acquired at intervals.
Optionally, in order to make the abnormal driving behavior detection more accurate, the shooting angle of the driver image may capture the front image of the driver as much as possible.
In some embodiments, prior to acquiring the driver image, further comprising:
The current running state of the driving tool is obtained, if the current running state comprises running, the image of the driver is obtained, so that resource waste caused by continuously shooting the driver when the driving tool is in a stop state and false detection caused by shooting reasonable actions of the driver when the driving tool is in a brake state can be avoided.
S102: and identifying human body key points of the driver in the driver image and position information of the human body key points based on the human body key point model.
The human body key point model can be a pre-trained identification model, and the human body key point model can adopt the identification model in the existing related technology.
Alternatively, the human body key point model can be obtained through training in the following way:
Acquiring M human body key points in a human body key point image, and converting each human body key point into a maximum activation point in an N matrix through Gaussian distribution;
Acquiring a backbone network, inputting a human body key point image into the backbone network, and generating an output result;
and determining L2 loss as a loss function according to the maximum activation point and the output result, and training the backbone network.
The human body key point image is an image marked with human body key points in advance, the positions of the human body key points can be determined according to the positions of human bones, see fig. 2 and 3, fig. 3 is a schematic diagram of a human body key point image, and fig. 2 is a schematic diagram of human body key points determined according to human bone joint points.
It should be noted that the serial numbers in fig. 2 are only examples, and those skilled in the art may change the identification manner of the serial numbers according to need, including but not limited to, letter, text, etc.
The manner of identifying human body key points in fig. 2 is also merely an example, and those skilled in the art may add other feature points as human body key points or delete part of human body key points as required.
Optionally, the backbone network includes, but is not limited to, a network mobilenetv, taking the human body key points shown in fig. 3 as an example, converting 21 human body key points of a human body into maximum activated points in a matrix of 64 x 64 through gaussian distribution, obtaining GT HEATMAP of 21 x 64, inputting an image of the human body key points into the network mobilenetv2, obtaining an output result of 21 x 64, determining L2 loss as a loss function according to the output result and GT HEATMAP of 21 x 64, and optimizing the network mobilenetv2 through the loss function.
Alternatively, the loss function may be determined based on the distance of the maximum activation point to the corresponding point mapped on the output result.
By the method of the feature map, the fact that the human body feature points have the correlated features is considered, the trained human body key point model can learn context semantic information in the image, overfitting can be effectively prevented, and more accurate human body key points are predicted.
The human body key points of the driver and the position information of the human body key points in the driver image can be accurately identified based on the human body key point model.
S103: if the identified human body key point set comprises the target human body key points, acquiring the position relation among the target human body key points, and detecting abnormal driving behaviors.
It should be noted that the target human body key points include human body key points of at least two preset positions.
Alternatively, the determination of whether the human body key point includes the target human body key point may be performed by identifying information of the human body key point, including, but not limited to, the serial numbers shown in fig. 2 and the like.
In some embodiments, if the identified human body key point set does not include the target human body key point, it is indicated that a part of the human body parts of the driver may be located outside the shooting area, for example, the head is extended out of the window, the hands are placed outside the shooting area, the driver is low, the driver is looking for an object falling into the vehicle by probing the hands, and so on. At this time, the driver's actions are all abnormal driving actions, and the detection result of the abnormal driving behavior detection is that there is abnormal driving behavior.
Alternatively, the determination of the target human body key points may be determined by a person skilled in the art according to the actual application scenario of the detection method.
In some embodiments, the target human body key points include a hand key point and an ear key point, and acquiring the positional relationship between the target human body key points includes:
determining a hand-ear distance between the hand key point and the ear key point according to the position information of the hand key point and the position information of the ear key point;
If the hand-ear distance is smaller than the preset hand-ear distance, selecting a hand-ear partial image from the driver image, wherein the hand-ear partial image comprises images of hand key points and positions of ear key points;
And carrying out image recognition on the partial images of the ears, and if the target object is recognized, determining that abnormal driving behaviors exist.
The hand keypoints comprise at least one of left hand keypoints and right hand keypoints, and the ear keypoints comprise at least one of left ear keypoints and right ear keypoints.
Optionally, the hand-ear distance includes, but is not limited to, at least one of:
A first ear distance from the left ear to the left hand, a second ear distance from the left ear to the right hand, a third ear distance from the right ear to the left hand, and a fourth ear distance from the right ear to the right hand.
In some embodiments, the method of image recognition of the partial image of the ear may employ image recognition methods in related art to identify the target object.
Optionally, the target items include, but are not limited to, cell phones and the like.
When the distance between the hands and the ears is smaller, it is likely that the driver is talking with the mobile phone, and the mobile phone talking has a larger influence on the driving safety, so that the abnormal driving behavior can be accurately detected by the method.
It should be noted that the preset ear-to-hand distance may be set according to the needs of those skilled in the art, and is not limited herein.
In some embodiments, the preset hand-ear distance can be 'different from person to person' according to the body condition of the driver, and by collecting the images of various normal driving actions of the driver in advance, the unreasonable hand-ear distance is determined as the preset hand-ear distance, so that system misjudgment caused by the adoption of a preset value due to roughness is avoided, and the accuracy and reliability of detection are improved.
In some embodiments, the identification of the target item may also be determined by determining whether an item is present in the driver's hand closer to the ear, and if so, whether it is identified as a cell phone or not, the item is considered the target item. In some cases, the driver may be performing an operation such as a belt, and the mobile phone cannot be recognized, but the behavior is still dangerous driving behavior, and thus, it may be defined that the object is the target object if the object is detected on the hand.
In some embodiments, the hand-ear partial image is a hand and ear corresponding to a hand-ear distance less than a preset hand-ear distance, and is not an entire facial image, for example, if it is detected that the distance from the left hand to the left ear is less than the preset hand-ear distance, the hand-ear partial image need only include the left hand and the left ear, and may not include at least one of the right ear or the right hand.
In some embodiments, referring to fig. 3, although the distance between the hands and the ears in fig. 3 is smaller than the preset distance between the hands and ears, because the hands of the driver do not have articles, no abnormal driving behavior exists, so that false detection caused by normal operations such as lifting the hands and stroking the hair of the driver can be effectively avoided, and the detection accuracy is further improved.
Referring to fig. 4, fig. 4 is an example of a partial image of the left ear, when a left-hand key point is detected near the left-hand key point, that is, when the first hand-ear distance from the left hand to the left ear is detected to be smaller than the preset hand-ear distance, the area where the left hand and the left ear are located is intercepted, after the partial image of the left ear is normalized, for example, the partial image of the hand ear is resized to 224x224, a detection method SSD (Single shotmultibox detector) is continuously used to detect whether a mobile phone is contained therein, and if the mobile phone is detected, it is determined that a phone calling behavior exists, that is, an abnormal driving behavior exists.
In some embodiments, the target human body key points include a hand key point and a lip key point, and acquiring the positional relationship between the target human body key points includes:
Determining a hand-lip distance between the hand key point and the lip key point according to the position information of the hand key point and the position information of the lip key point;
If the hand-lip distance is smaller than the preset hand-lip distance, selecting a hand-lip partial image from the driver image, wherein the hand-lip partial image comprises images of hand key points and positions of the lip key points;
and carrying out image recognition on the partial images of the hand lips, and if the slender object is recognized, determining that abnormal driving behaviors exist.
The hand keypoints include at least one of a left hand keypoint and a right hand keypoint.
Optionally, the hand-ear distance includes, but is not limited to, at least one of:
Lip to left hand first hand lip distance, lip to right hand second hand lip distance.
In some embodiments, the manner in which the partial images of the lips are image-identified may employ image-identification methods of the related art to identify the elongate article.
Optionally, the elongate article includes, but is not limited to, a cigarette, an electronic cigarette, and the like.
When the distance between the hands and lips is small, it is likely that the driver is smoking, and smoking has a large influence on driving safety, so that such abnormal behavior can be accurately detected by the method.
The preset distance between the lips may be set according to the needs of those skilled in the art, and is not limited herein.
In some embodiments, the preset hand-lip distance can be 'different from person to person' according to the body condition of the driver, and an unreasonable hand-lip distance is determined as the preset hand-lip distance by collecting the images of various normal driving actions of the driver in advance, so that system misjudgment caused by the adoption of a preset value due to roughness is avoided, and the accuracy and reliability of detection are improved.
In some embodiments, the elongate article may also be identified by determining whether an article is present in the driver's hand closer to the lip, and if so, whether it is identified as an elongate article or not, the article is considered to be an elongate article. In some cases, the driver may be eating something or the like, and although the driver cannot recognize the elongated article, the behavior is still dangerous driving behavior, and thus, it may be defined that the elongated article is the elongated article as long as the existence of the elongated article is detected by the hand.
In some embodiments, the partial image of the hand and the lip is a partial image of the hand and the lip corresponding to a distance between the hand and the lip less than a preset distance between the hand and the lip, instead of the whole facial image, for example, if the distance between the left hand and the lip is detected to be less than the preset distance between the hand and the lip, the partial image of the hand and the lip only needs to include the left hand and the lip, and may not need to include the right hand.
In some embodiments, determining that abnormal driving behavior exists includes:
Respectively obtaining a first pixel average value of one end of the elongated object far away from the lip and a second pixel average value of a partial image of the hand lip;
If the first pixel average value is larger than the second pixel average value, determining that abnormal driving behaviors exist.
When the distance between the hands and lips of the driver is too small, the driver can drink water and eat something, and the driver can send out voice messages or smoke, and the like, so that compared with the prior art, the smoke smoking process is longer in duration and more remarkable in danger. The electronic cigarette or the cigarette may have a light emitting state at the end far from the lip during smoking, so that the first pixel average value of the area where the end of the elongated object far from the lip is located can be compared with the second pixel average value of the whole shou chu partial image, if the first pixel average value is greater than the second pixel average value, it is indicated that the end of the elongated object is in the light emitting state, and it is highly likely that the driver is smoking at the moment, so that it is determined that abnormal driving behavior exists.
Of course, the cigarette may also have a state that the driver spits out smoke, so that in order to further determine whether the driver is smoking, a partial image of the lip of the driver may be further obtained, whether the smoke exists is identified by an image identification mode, and if the smoke exists, abnormal driving behavior may be further determined.
Referring to fig. 5, fig. 5 is an example of a partial image of the hand and the lip, when a left hand key point is detected near a lip key point, that is, when a first hand-lip distance from the left hand to the lip is detected to be smaller than a preset hand-lip distance, a region where the left hand and the lip are located is intercepted, the partial image of the hand and the lip is normalized, for example, the partial image of the hand and the lip is resize to 224x224, and whether a cigarette is contained in the partial image of the hand and the lip is detected by using a SSD (Single shotmultibox detector) detection method is continuously, if the cigarette is detected, the smoking behavior is judged to exist, that is, abnormal driving behavior exists.
In some embodiments, to improve the accuracy of abnormal driving behavior detection, at least one image of a preset time before or after the current driver image may be further acquired based on the determination that the abnormal driving behavior exists based on the current driver image, further at least one abnormal driving behavior detection may be performed, and if the detection results exceeding a certain proportion are consistent, it is determined that the abnormal driving behavior does exist. For example, the driver image currently acquired is 10:00:00, if abnormal driving behaviors are detected based on the image, respectively acquiring 10:00:03 and 9:59:58, and detecting that the driver has abnormal driving behaviors in at least one of the images, and further determining that the abnormal driving behaviors exist.
In some embodiments, the abnormal driving behavior detection method further includes:
the target key points comprise hand key points, and the hand key points comprise left hand key points and right hand key points;
And acquiring a hand distance between the left hand key point and the right hand key point, acquiring a hand partial image if the hand distance is not within a preset hand distance range threshold, identifying the hand partial image, determining that abnormal behavior is normal if the hand partial image is identified on the steering wheel, and determining that abnormal driving behavior exists if the hand partial image is not within the preset hand distance range threshold.
When the driving tool is an automatic gear vehicle, gear shifting is not needed in the driving process, so that a driver is usually required to hold the steering wheel by both hands in the driving process of the motor vehicle, and bad habit that both hands leave the steering wheel can exist when the vehicle is in a straight line driving at a constant speed by some drivers, therefore, if the hand distance is detected, the driver may not place both hands on the steering wheel if the hand key points are not detected, or the distance between the hand key points is too large or too small, the driver may not place both hands on the steering wheel, at the moment, danger is possibly caused to driving, whether the hands are placed on the steering wheel is determined through image recognition, and abnormal driving behavior that both hands leave the steering wheel can be effectively detected.
In some embodiments, the target keypoints include a left-hand keypoint and a right-hand keypoint, the obtaining a positional relationship between the target human body keypoints, and the performing abnormal driving behavior detection includes:
Determining the hand distance between two hands according to the position information of the left hand key point and the position information of the right hand key point;
Acquiring a hand partial image, identifying a steering wheel in the hand partial image, and determining the diameter of the steering wheel;
if the hand distance is smaller than the diameter, determining that abnormal driving behaviors exist.
In life, the driving habit of some people is very bad, is used to holding the double hands in the lower half area of the steering wheel, and the direction of the double hands is often not adjusted when an emergency occurs, so that dangerous actions can be timely detected through the mode, and related people can be timely reminded, so that accidents are avoided.
In some embodiments, if two hands are not successfully detected in the steering wheel region, further determining a hand-ear distance between the hand key point and the ear key point according to the position information of the hand key point and the position information of the ear key point, and/or determining a hand-lip distance between the hand key point and the lip key point according to the position information of the hand key point and the position information of the lip key point, so as to correspondingly execute the corresponding detection steps.
In some embodiments, the abnormal driving behavior detection method further includes:
Acquiring position information of a current human body key point of a face, and selecting a face partial image;
Identifying facial key points in the facial partial image and position information of the facial key points based on the facial key point model;
determining a deflection angle according to the position information of the facial key points;
And if the deflection angle is larger than the preset deflection angle threshold value, determining that abnormal driving behaviors exist.
In some embodiments, after selecting the face partial image, before identifying the face key points in the face partial image and the position information of the face key points based on the face key point model, further comprising:
and carrying out normalization processing on the partial facial image.
Optionally, after the partial facial image and the image with the size of 112x112 are input into the face key point model, the coordinate information of the face key points can be obtained.
In some embodiments, the face keypoint model may be an existing correlation model, and one possible recognition result may be as shown in fig. 6. A plurality of recognition points are arranged according to the positions of facial features and contours.
In some embodiments, the number of face keypoints output by the face keypoint model is 68, the distribution of which may be seen in fig. 6.
In some embodiments, the yaw angle includes at least one of pitch angle, roll angle, yaw angle. When the deflection angle comprises at least two of a pitch angle, a roll angle and a yaw angle, the driver can be judged to have a deviation realization behavior when one of the angles is larger than a preset deflection angle threshold value, and abnormal driving behaviors can be determined.
Alternatively, the yaw angle is obtained by:
θ x＝atan2(-ryz,rzz) equation (1).
Alternatively, the roll angle is obtained by:
alternatively, the pitch angle is obtained by:
θ z＝atan2(-rxy,rxx) equation (3).
Wherein, θ x is yaw angle, θ y is roll angle, and θ z is pitch angle.
Wherein r yz、rzz、rrz、rxy、rxx can be determined according to the following manner:
yaw angle Yaw is the angle of rotation about the Y-axis, pitch angle pitch is the angle of rotation about the x-axis, roll angle roll is the angle of rotation about the z-axis, and then there are:
Rotated in succession by the yaw-pitch-roll, the matrix is described as:
According to the position information of each face key point, r yz、rzz、rrz、rxy、rxx can be determined, and then the deflection angle is determined.
In some embodiments, the abnormal driving behavior detection method further includes:
And if the abnormal driving behavior is determined, sending out an alarm message.
Optionally, the alarm message may be displayed by means of an alarm bell, an alarm lamp and the like carried by the driving tool, and the alarm message may also be transmitted to a corresponding message receiving terminal by means of wired or wireless communication, where the message receiving terminal includes, but is not limited to, a mobile terminal (mobile phone and the like) carried by the driver, a third party platform (such as a traffic police monitoring platform, a driver management platform) and the like.
In some embodiments, the abnormal driving behavior detection method further includes:
and acquiring abnormal driving behavior information, and determining the risk degree of abnormal driving according to a preset rule.
The abnormal driving behavior information includes, but is not limited to, abnormal driving duration, geographical location information of a place where the abnormal driving behavior occurs, running speed of a current driving tool, road section state where the current driving tool is located, specific type of the abnormal behavior, and the like.
The road segment status in which the driving tool is currently located includes, but is not limited to, at least one of long straight roads, multi-turn roads, single-way roads, downhill slopes, uphill slopes, etc.
Specific types of abnormal behavior include, but are not limited to, at least one of smoking, making a call, eating a snack, going away from mind, etc.
The preset rule may be to preset a certain weight score for various abnormal driving behavior information, further determine a comprehensive score, and determine the risk degree of the abnormal driving behavior according to the magnitude of the comprehensive score. The risk level includes, but is not limited to: very dangerous, more dangerous, generally dangerous, etc.
For example, the current abnormal driving behavior information includes turning, long downhill slope, going away for 3 seconds, the total score is determined to be 80 according to the preset weight score, and the preset total score is set to be very dangerous in excess of 70. The degree of risk of abnormal driving at this time is very dangerous.
Alternatively, the warning frequency and the warning mode may be determined according to the risk degree of abnormal driving.
The embodiment of the invention provides an abnormal driving behavior detection method, which comprises the steps of identifying human body key points of a driver in a driver image and position information of the human body key points according to a human body key point model by acquiring the driver image, and if the identified human body key point set comprises target human body key points, acquiring the position relation among the target human body key points to detect the abnormal driving behavior. The problem that the abnormal driving behavior is difficult to detect when the driving tool only has one driver is avoided, the detection speed and accuracy of the abnormal driving behavior are improved, and the driving safety can be effectively improved.
Example two
Referring to fig. 7, an abnormal driving behavior detection apparatus 1000 includes:
an acquisition module 1001 for acquiring a driver image;
The identifying module 1002 is configured to identify a human body key point of a driver in the driver image and position information of the human body key point based on the human body key point model;
The detection module 1003 is configured to, if the identified set of human body keypoints includes a target human body keypoint, obtain a positional relationship between the target human body keypoints, detect abnormal driving behavior, and the target human body keypoints include human body keypoints of at least two preset positions.
In this embodiment, the abnormal driving behavior detection device is substantially provided with a plurality of modules for executing the abnormal driving behavior detection method in the above embodiment, and specific functions and technical effects may be described with reference to the above embodiment, which is not repeated herein.
Referring to fig. 8, an embodiment of the present invention further provides a terminal 1100, including a processor 1101, a memory 1102, and a communication bus 1103;
A communication bus 1103 is used to connect the processor 1101 and the memory connection 1102;
The processor 1101 is configured to execute a computer program stored in the memory 1102 to implement the abnormal driving behavior detection method according to any one of the above embodiments.
An embodiment of the invention also provides a computer-readable storage medium, characterized in that it has stored thereon a computer program,
The computer program is for causing a computer to execute the abnormal driving behavior detection method according to any one of the above embodiments.
The embodiment of the application also provides a non-volatile readable storage medium, where one or more modules (programs) are stored, where the one or more modules are applied to a device, and the instructions (instructions) may cause the device to execute the steps included in the embodiment one of the embodiment of the application.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (8)
1. An abnormal driving behavior detection method, characterized by comprising:
acquiring an image of a driver;
Identifying human body key points of a driver in the driver image and position information of the human body key points based on a human body key point model;
If the identified human body key point set comprises target human body key points, acquiring the position relation among the target human body key points, and detecting abnormal driving behaviors, wherein the target human body key points comprise human body key points of at least two preset parts;
The target human body key points comprise hand key points and lip key points, the obtaining of the position relation between the target human body key points comprises the following steps of:
determining a hand-lip distance between a hand key point and a lip key point according to the position information of the hand key point and the position information of the lip key point;
if the hand-lip distance is smaller than the preset hand-lip distance, selecting a hand-lip partial image from the driver image, wherein the hand-lip partial image comprises images of hand key points and positions of lip key points;
performing image recognition on the partial images of the hand lips, and if the slender object is recognized, determining that abnormal driving behaviors exist;
The target human body key points comprise hand key points and ear key points, the obtaining of the position relation between the target human body key points comprises:
determining a hand-ear distance between the hand key point and the ear key point according to the position information of the hand key point and the position information of the ear key point;
If the hand-ear distance is smaller than the preset hand-ear distance, selecting a hand-ear partial image from the driver image, wherein the hand-ear partial image comprises images of hand key points and positions of ear key points;
And carrying out image recognition on the hand-ear partial images, and if the mobile phone is recognized, determining that abnormal driving behaviors exist.
2. The abnormal driving behavior detection method according to claim 1, wherein determining that abnormal driving behavior exists includes:
Respectively obtaining a first pixel average value of one end of the slender object far away from the lip and a second pixel average value of the partial image of the hand lip;
If the first pixel average value is larger than the second pixel average value, determining that abnormal driving behaviors exist.
3. The abnormal driving behavior detection method according to claim 1, characterized by further comprising:
Acquiring position information of a current human body key point of a face, and selecting a face partial image;
identifying facial key points in the facial partial image and position information of the facial key points based on a facial key point model;
Determining a deflection angle according to the position information of the facial key points, wherein the deflection angle comprises at least one of a pitch angle, a roll angle and a yaw angle;
and if the deflection angle is larger than a preset deflection angle threshold value, determining that abnormal driving behaviors exist.
4. The abnormal driving behavior detection method according to claim 3, wherein,
The yaw angle is obtained by:
θx＝atan2(-ryz,rzz)；
the roll angle is obtained by:
the pitch angle is obtained by:
θz＝atan2(-rxy,rxx)；
wherein, θ x is yaw angle, θ y is roll angle, and θ z is pitch angle.
5. The abnormal driving behavior detection method according to any one of claims 1 to 4, wherein the training method of the human body key point model comprises:
Obtaining M human body key points in a human body key point image, wherein each human body key point is converted into a maximum activation point in an N matrix through Gaussian distribution;
acquiring a backbone network, inputting the human body key point image into the backbone network, and generating an output result;
and determining L2 loss as a loss function according to the maximum activation point and the output result, and training the backbone network.
6. An abnormal driving behavior detection device, characterized by comprising:
The acquisition module is used for acquiring the image of the driver;
The identification module is used for identifying human body key points of a driver in the driver image and position information of the human body key points based on a human body key point model;
The detection module is used for acquiring the position relation between the target human body key points if the identified human body key point set comprises the target human body key points, and detecting abnormal driving behaviors, wherein the target human body key points comprise human body key points of at least two preset parts;
wherein, the target human body key points comprise a hand key point and a lip key point, and the detection module is further used for:
determining a hand-lip distance between a hand key point and a lip key point according to the position information of the hand key point and the position information of the lip key point;
if the hand-lip distance is smaller than the preset hand-lip distance, selecting a hand-lip partial image from the driver image, wherein the hand-lip partial image comprises images of hand key points and positions of lip key points;
performing image recognition on the partial images of the hand lips, and if the slender object is recognized, determining that abnormal driving behaviors exist;
Wherein, the human keypoints of target include hand keypoints, ear keypoints, detection module still is used for:
determining a hand-ear distance between the hand key point and the ear key point according to the position information of the hand key point and the position information of the ear key point;
If the hand-ear distance is smaller than the preset hand-ear distance, selecting a hand-ear partial image from the driver image, wherein the hand-ear partial image comprises images of hand key points and positions of ear key points;
And carrying out image recognition on the hand-ear partial images, and if the mobile phone is recognized, determining that abnormal driving behaviors exist.
7. A terminal comprising a processor, a memory and a communication bus;
the communication bus is used for connecting the processor and the memory;
the processor is configured to execute a computer program stored in the memory to implement the abnormal driving behavior detection method according to any one of claims 1 to 5.
8. A computer-readable storage medium, having a computer program stored thereon,
The computer program is for causing the computer to execute the abnormal driving behavior detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110349516.7A CN112990069B (en) | 2021-03-31 | 2021-03-31 | Abnormal driving behavior detection method, device, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110349516.7A CN112990069B (en) | 2021-03-31 | 2021-03-31 | Abnormal driving behavior detection method, device, terminal and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990069A CN112990069A (en) | 2021-06-18 |
| CN112990069B true CN112990069B (en) | 2024-10-15 |
Family
ID=76338713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110349516.7A Active CN112990069B (en) | 2021-03-31 | 2021-03-31 | Abnormal driving behavior detection method, device, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990069B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113470080B (en) * | 2021-07-20 | 2024-05-14 | 浙江大华技术股份有限公司 | Illegal behavior recognition method |
| CN113591661B (en) * | 2021-07-24 | 2024-10-29 | 深圳市铁越电气有限公司 | Method and system for predicting call making behavior |
| CN114155595A (en) * | 2021-09-30 | 2022-03-08 | 深圳市爱深盈通信息技术有限公司 | Behavior detection monitoring method, intelligent camera and intelligent monitoring system |
| CN116363632A (en) * | 2021-12-23 | 2023-06-30 | 比亚迪股份有限公司 | Method and device for monitoring abnormal behavior in vehicle and computer storage medium |
| CN114663863A (en) * | 2022-02-24 | 2022-06-24 | 北京百度网讯科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN114241589B (en) * | 2022-02-28 | 2022-08-23 | 深圳市城市交通规划设计研究中心股份有限公司 | Bus driver violation behavior determination method and device based on vehicle-mounted video |
| CN120199080A (en) * | 2025-05-26 | 2025-06-24 | 湖南车路协同智能科技有限公司 | A road safety analysis method based on vehicle-road collaboration and related device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111301280A (en) * | 2018-12-11 | 2020-06-19 | 北京嘀嘀无限科技发展有限公司 | Dangerous state identification method and device |
| CN111368696A (en) * | 2020-02-28 | 2020-07-03 | 淮阴工学院 | Detection method and system for illegal driving behavior of hazardous chemicals transport vehicle based on visual collaboration |
| CN112287795A (en) * | 2020-10-22 | 2021-01-29 | 北京百度网讯科技有限公司 | Abnormal driving posture detection method, device, equipment, vehicle and medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109803583A (en) * | 2017-08-10 | 2019-05-24 | 北京市商汤科技开发有限公司 | Driver monitoring method, apparatus and electronic equipment |
| CN107697069B (en) * | 2017-10-31 | 2020-07-28 | 上海汽车集团股份有限公司 | Intelligent control method for fatigue driving of automobile driver |
| CN110956061B (en) * | 2018-09-27 | 2024-04-16 | 北京市商汤科技开发有限公司 | Action recognition method and device, and driver state analysis method and device |
| CN110516620B (en) * | 2019-08-29 | 2023-07-28 | 腾讯科技（深圳）有限公司 | Target tracking method and device, storage medium and electronic equipment |
| CN110837815A (en) * | 2019-11-15 | 2020-02-25 | 济宁学院 | Driver state monitoring method based on convolutional neural network |
| CN111488855A (en) * | 2020-04-24 | 2020-08-04 | 上海眼控科技股份有限公司 | Fatigue driving detection method, device, computer equipment and storage medium |
| CN112016457A (en) * | 2020-08-27 | 2020-12-01 | 青岛慕容信息科技有限公司 | Driver distraction and dangerous driving behavior recognition method, device and storage medium |
| CN112069988A (en) * | 2020-09-04 | 2020-12-11 | 徐尔灵 | Gun-ball linkage-based driver safe driving behavior detection method |
| CN112084719B (en) * | 2020-09-17 | 2022-08-02 | 招商局重庆交通科研设计院有限公司 | Road traffic intelligent design platform based on computer aided design and auxiliary simulation |
| CN112183482A (en) * | 2020-10-29 | 2021-01-05 | 苏州市职业大学 | Dangerous driving behavior identification method, device, system and readable storage medium |
| CN112270283A (en) * | 2020-11-04 | 2021-01-26 | 北京百度网讯科技有限公司 | Abnormal driving behavior determination method, device, equipment, vehicle and medium |
-
2021
- 2021-03-31 CN CN202110349516.7A patent/CN112990069B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111301280A (en) * | 2018-12-11 | 2020-06-19 | 北京嘀嘀无限科技发展有限公司 | Dangerous state identification method and device |
| CN111368696A (en) * | 2020-02-28 | 2020-07-03 | 淮阴工学院 | Detection method and system for illegal driving behavior of hazardous chemicals transport vehicle based on visual collaboration |
| CN112287795A (en) * | 2020-10-22 | 2021-01-29 | 北京百度网讯科技有限公司 | Abnormal driving posture detection method, device, equipment, vehicle and medium |
Non-Patent Citations (2)
| Title |
|---|
| 姚巍巍 ; 张洁 ; .融合目标检测和人体关键点检测的铁路司机行为识别.计算机测量与控制.2020,(第06期),第2节第1段，第2.3节第1、2段. * |
| 融合目标检测和人体关键点检测的铁路司机行为识别;姚巍巍;张洁;;计算机测量与控制(第06期);第2节第1段，第2.3节第1、2段 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990069A (en) | 2021-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112990069B (en) | Abnormal driving behavior detection method, device, terminal and medium | |
| CN107608388B (en) | Autonomous police vehicle | |
| CN109345829B (en) | Unmanned vehicle monitoring method, device, equipment and storage medium | |
| CN111210620B (en) | Method, device and equipment for generating driver portrait and storage medium | |
| US12106670B2 (en) | Accident reporter | |
| JP6827712B2 (en) | Control devices, in-vehicle devices, video distribution methods, and programs | |
| CN104853972A (en) | Augmenting ADAS features of vehicle with image processing support in on-board vehicle platform | |
| CN110991353A (en) | Early warning method for recognizing driving behaviors of driver and dangerous driving behaviors | |
| CN205281786U (en) | alarm system | |
| CN111985373A (en) | Safety warning method, device and electronic device based on traffic intersection recognition | |
| WO2017190497A1 (en) | Vehicle alerting method and device | |
| US20220358835A1 (en) | Methods and systems for automatically detecting violation of a driving-related law | |
| CN113352989B (en) | Intelligent driving safety auxiliary method, product, equipment and medium | |
| CN114664085A (en) | Dangerous road section reminding method and device, electronic equipment and medium | |
| CN112633387A (en) | Safety reminding method, device, equipment, system and storage medium | |
| JP2014164375A (en) | Information processor, information processing system and information processing method, and program for information processor | |
| CN107680396A (en) | The monitoring method and its device of a kind of driving behavior | |
| CN116798247A (en) | Accident data prompting method and related equipment | |
| EP4586227A1 (en) | Vehicle early warning method and apparatus, device, storage medium, and product | |
| CN114103944A (en) | Workshop time distance adjusting method, device and equipment | |
| CN113689719A (en) | Vehicle state determination method, vehicle state determination device, electronic device, and storage medium | |
| CN114037763A (en) | Target object missing detection method and device, storage medium and electronic equipment | |
| CN117854300A (en) | Safety behavior early warning method based on driver dangerous driving behavior assessment | |
| CN112435475B (en) | Traffic state detection method, device, equipment and storage medium | |
| JP7470967B2 (en) | Systems, programs, machine learning methods, and machine learning models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240913 Address after: 203B, Building 9, Phase II, Nanshan Yungu Entrepreneurship Park, No. 2 Pingshan 1st Road, Pingshan Community, Taoyuan Street, Nanshan District, Shenzhen City, Guangdong Province 518000 Applicant after: Core Computing Integrated (Shenzhen) Technology Co.,Ltd. Country or region after: China Address before: Room 1010-1011, 10 / F, building a, Chuangzhi building, Xinjiang Software Park, 455 Kanas Hubei Road, Urumqi Economic and Technological Development Zone, Urumqi City, Xinjiang Uygur Autonomous Region 830023 Applicant before: XINJIANG AIHUA YINGTONG INFORMATION TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |