CN110677592A - Subject focusing method and device, computer equipment and storage medium - Google Patents

Abstract

The application relates to a main body focusing method, a main body focusing device, computer equipment and a storage medium, wherein attitude information of each main body is acquired from a focusing visual field, special attitude identification or motion degree identification made by each main body can be estimated through the attitude information of the main body, and a target main body is determined based on a preset screening rule, so that the target main body can be effectively focused when a picture is taken or a video is recorded, the method can be used for automatically focusing on a person of the target main body instead of manually focusing, the focusing flexibility and accuracy are improved, and the use experience is improved. In addition, the method can improve the capture of the posture and the motion of the main body by the focusing function of the camera, so that the focusing can understand the behavior of people more intelligently, and the use interestingness is improved.

Description

Subject focusing method, apparatus, computer equipment and storage medium

technical field

The present application relates to the technical field of image processing, and in particular, to a subject focusing method, apparatus, computer equipment and storage medium.

Background technique

With the popularity of smartphones, taking photos and video recordings through mobile phones has become the norm.

The current focusing methods for taking pictures and video recordings of mobile phones include: focusing by manually touching the screen by the user and focusing according to the default center area of the screen. The focusing functions of existing mobile phones for taking pictures and video recordings have the problems of poor flexibility and inaccurate focusing.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide a subject focusing method, device, computer device and storage medium in response to the above technical problems.

In a first aspect, an embodiment of the present application provides a method for focusing a subject, the method comprising:

Obtain the subject posture information of the subject in the focused field of view;

According to the subject posture information and the preset screening rules, the target subject is determined from the focused field of view; the screening rules are rules determined according to the gesture action and/or the degree of movement;

Focus on the target subject.

In one of the embodiments, the above-mentioned screening rules include gesture action rules and/or motion degree rules;

Gesture action rules are used to indicate the priority of each gesture action;

The motion level rules are used to indicate the priority of each motion level.

In one embodiment, the above-mentioned determination of the target subject from the focused field of view according to the subject posture information and the preset screening rules includes:

According to the subject posture information, determine the posture action of each subject in the focused field of view;

According to the gesture action rules, the subject with the highest priority in each gesture action is determined as the target subject.

In one embodiment, the above-mentioned determination of the posture actions of each subject in the focused field of view according to the subject posture information includes:

According to the subject's posture information, the angle between each subject's limbs is obtained;

From the preset posture action information table, the posture action corresponding to each main body limb is determined; the posture action information table includes the correspondence between the angles between the plurality of main body limbs and the posture action.

In one embodiment, the above-mentioned determination of the target subject from the focused field of view according to the subject posture information and the preset screening rules includes:

According to the subject posture information, obtain the movement degree of each subject in the focus field of view in consecutive N frames;

According to the motion degree rules, the subject with the highest priority in each motion degree is determined as the target subject.

In one of the embodiments, the above-mentioned acquisition of the motion degree of each subject in the focused field of view in consecutive N frames according to the subject posture information includes:

According to the subject's posture information, obtain the angle change of each subject's limb key points in consecutive N frames;

From the preset motion degree information table, determine the motion degree corresponding to the angle change of each subject's limb key point; the motion degree information table includes the corresponding relationship between the angle change of the main body's limb key point and the motion degree.

In one embodiment, the method includes: if the target subject does not exist in the focused field of view, or if the focus tracking duration of the target subject reaches a preset time period, re-determining a new target subject from the focused field of view.

In one embodiment, the above-mentioned acquisition of subject posture information of the subject in the focused field of view includes:

Detect the key bone information of each subject in the focused field of view;

According to the key information of the skeleton, the posture information of each subject is determined.

In one embodiment, the method further includes:

If the target subject is not determined from the focus field of view, the subject is focused according to the central area of the focus field of view.

In a second aspect, an embodiment of the present application provides a main body focusing device, the device comprising:

an acquisition module, used to acquire the subject posture information of the subject in the focused field of view;

a determination module, used for determining the target subject from the focus field of view according to the subject posture information and preset screening rules; the screening rules are rules determined according to the gesture action and/or the degree of movement;

The focus tracking module is used to track the target subject.

In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of any one of the methods provided in the first embodiment of the first aspect when the processor executes the computer program.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any one of the methods provided in the embodiments of the first aspect.

In a subject focusing method, device, computer equipment, and storage medium provided by the embodiments of the present application, the subject's posture information is obtained from the focusing field of view, and the special posture recognition or movement degree made by each subject can be estimated through the subject's posture information. and based on the preset screening rules, the target subject is determined, so that the target subject can be effectively focused when taking pictures or videos. This method is used to automatically focus on the person of the target subject instead of manual focus, which improves the The flexibility and accuracy of focusing improves the user experience. In addition, this method can improve the camera's focus function to capture the subject's posture and motion, so that the focus can more intelligently understand human behavior and improve the fun of use.

Description of drawings

1 is a schematic diagram of an image processing circuit provided by an embodiment;

2 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

3 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

3a is a schematic diagram of a human body posture action provided by an embodiment;

4 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

5 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

6 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

7 is a schematic flowchart of a method for focusing a subject provided by an embodiment;

Fig. 7a is a schematic diagram of key bone information provided by an embodiment;

8 is a complete schematic diagram of a subject focusing method provided by an embodiment;

9 is a complete schematic diagram of a subject focusing method provided by an embodiment;

10 is a structural block diagram of a main body focusing device provided by an embodiment;

11 is a structural block diagram of a main body focusing device provided by an embodiment;

12 is a structural block diagram of a main body focusing device provided by an embodiment;

Figure 13 is a diagram of the internal structure of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The subject focusing method provided by the present application can be applied to electronic equipment. The electronic device may be a computer device with a camera, a personal digital assistant, a tablet computer, a smart phone, a wearable device, or the like. When a camera in an electronic device captures an image, it will automatically focus to ensure that the captured image is clear.

In one embodiment, the above electronic device may include an image processing circuit, which may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. FIG. 1 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 1 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 1 , the image processing circuit includes a first ISP processor 130 , a second ISP processor 140 and a control logic 150 . The first camera 110 includes one or more first lenses 112 and a first image sensor 114 . The first image sensor 114 may include a color filter array (eg, a Bayer filter), the first image sensor 114 may acquire light intensity and wavelength information captured with each imaging pixel of the first image sensor 114, and provide information that can be accessed by the first ISP A set of image data processed by processor 130 . The second camera 120 includes one or more second lenses 122 and a second image sensor 124 . The second image sensor 124 may include a color filter array (eg, a Bayer filter), the second image sensor 124 may obtain light intensity and wavelength information captured with each imaging pixel of the second image sensor 124, and provide information that can be accessed by the second ISP A set of image data processed by processor 140 .

The first image collected by the first camera 110 is transmitted to the first ISP processor 130 for processing. After the first ISP processor 130 processes the first image, the statistical data of the first image (such as the brightness of the image, the contrast value of the image, etc.) , image color, etc.) to the control logic 150, and the control logic 150 can determine the control parameters of the first camera 110 according to the statistical data, so that the first camera 110 can perform auto-focus, auto-exposure and other operations according to the control parameters. After being processed by the first ISP processor 130, the first image may be stored in the image memory 160, and the first ISP processor 130 may also read the image stored in the image memory 160 for processing. In addition, after being processed by the ISP processor 130, the first image can be directly sent to the display 170 for display, and the display 170 can also read the image in the image memory 160 for display.

Among them, the first ISP processor 130 processes the image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the first ISP processor 130 may perform one or more image processing operations on the image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

The image memory 160 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

Upon receiving the interface from the first image sensor 114, the first ISP processor 130 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 160 for additional processing before being displayed. The first ISP processor 130 receives processing data from the image memory 160 and performs image data processing in RGB and YCbCr color spaces on the processed data. The image data processed by the first ISP processor 130 may be output to the display 170 for viewing by the user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the first ISP processor 130 may also be sent to the image memory 160 , and the display 170 may read image data from the image memory 160 . In one embodiment, image memory 160 may be configured to implement one or more frame buffers.

Statistics determined by the first ISP processor 130 may be sent to the control logic 150 . For example, the statistics may include first image sensor 114 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, first lens 112 shading correction, and the like. The control logic 150 may include a processor and/or a microcontroller executing one or more routines (eg, firmware) that may determine the control parameters and the first camera 110 based on the received statistics. A control parameter of the ISP processor 130. For example, the control parameters of the first camera 110 may include gain, integration time for exposure control, anti-shake parameters, flash control parameters, first lens 112 control parameters (eg, focal length for focusing or zooming), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and first lens 112 shading correction parameters.

Similarly, the second image captured by the second camera 120 is transmitted to the second ISP processor 140 for processing. After the second ISP processor 140 processes the first image, the statistical data of the second image (such as the brightness of the image, the The contrast value, the color of the image, etc.) are sent to the control logic 150, and the control logic 150 can determine the control parameters of the second camera 120 according to the statistical data, so that the second camera 120 can perform automatic focusing, automatic exposure and other operations according to the control parameters. . After being processed by the second ISP processor 140, the second image can be stored in the image memory 160, and the second ISP processor 140 can also read the image stored in the image memory 160 for processing. In addition, after being processed by the ISP processor 140, the second image can be directly sent to the display 170 for display, and the display 170 can also read the image in the image memory 160 for display. The second camera 120 and the second ISP processor 140 can also implement the processing procedures as described for the first camera 110 and the first ISP processor 130 .

In one embodiment, the first camera 110 may be a color camera, and the second camera 120 may be a TOF (Time Of Flight) camera or a structured light camera. The TOF camera can obtain the TOF depth map, and the structured light camera can obtain the structured light depth map. The first camera 110 and the second camera 120 may both be color cameras. The binocular depth map is acquired by two color cameras. The first ISP processor 130 and the second ISP processor 140 may be the same ISP processor.

The first camera 110 and the second camera 120 capture the same scene to obtain a visible light image and a depth image, respectively, and send the visible light image and the depth image to the ISP processor. The ISP processor can register the visible light map and the depth map according to the camera calibration parameters to keep the field of view completely consistent; and then generate a center weight map corresponding to the visible light map, wherein the weight value represented by the center weight map is from the center to the edge. Gradually decrease; input the visible light map and the center weight map into the trained subject detection model to obtain the confidence map of the subject area, and then determine the target subject in the visible light map according to the confidence map of the subject area; you can also use the visible light map, depth The image and the center weight map are input into the trained subject detection model, and the confidence map of the subject area is obtained, and then the target subject in the visible light image is determined according to the confidence map of the subject area. Using the center weight map can make objects located in the center of the image easier to detect, and using the depth map can make objects closer to the camera easier to detect, improving the accuracy of subject detection.

The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems will be specifically described in detail below with reference to the accompanying drawings. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that, in a subject focusing method provided by the present application, the execution subject of FIG. 2 to FIG. 9 is an electronic device with a camera, wherein, the execution subject may also be a subject focusing device, wherein the device can use software, Hardware or a combination of software and hardware is implemented as part or all of an electronic device.

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments.

FIG. 2 is a flowchart of a method for focusing on a subject in an embodiment. As shown in FIG. 2, a method for focusing on a subject is provided, which can be applied to the electronic device in FIG. 1, including:

S101, acquiring subject posture information of a subject in a focused field of view.

The focus field of view refers to a field of view image displayed on the screen of the electronic device when taking a picture or recording a video. Among them, the subject represents the main part in the focused field of view, such as people, objects, plants, animals, etc. Generally, the focus field may include one subject, or may include multiple subjects, and the posture information of each subject is different. Wherein, the main body posture information may include position information of each main body limb, etc., and the present embodiment does not limit the posture information.

Specifically, taking the subject as a person as an example, the electronic device obtains the human body posture information of all people in the focused field of view, which may be obtained after the electronic device turns on the camera and before pressing the camera button/recording button. Human body posture information.

S102, according to the subject posture information and a preset screening rule, determine the target subject from the focused field of view; the screening rule is a rule determined according to the gesture action and/or the degree of movement.

When the electronic device detects that the focus field includes multiple subjects, a target subject can be selected from the multiple subjects for focusing. Specifically, the electronic device determines a target subject from a plurality of subjects according to the acquired posture information of each subject and a preset screening rule.

Among them, the screening rule is a rule determined according to the posture action of each subject or the degree of movement of each subject. Generally, the posture action can be determined according to the position of each limb in the posture information. For example, if both arms of subject A are open, then the subject The posture action of A can be to open his arms. For another example, if the two legs of the subject B are at a 90-degree angle, and one of the legs is parallel to the ground, the posture action of the subject B is that one leg is raised. Based on the posture action of the subject, when defining the screening rules of the posture action, the priority of the posture action can be used for screening. For example, if the priority of opening the arms is higher than that of raising the single leg, the subject with the arms open is selected during the screening. A as the target subject.

Among them, the degree of movement can be determined according to the range of changes in the position of the subject's limbs in consecutive frames, that is, the degree of movement represents the magnitude of the changes in the position of the subject's limbs in consecutive frames. In the middle, the angle between the arm and the leg of subject A is 30 degrees. In the next frame, the angle between the arm and the leg of subject A changes to 150 degrees, and the angle change amplitude between two consecutive frames is 120 degrees, and it is also determined that the angle change amplitude between two consecutive frames of subject B is 90 degrees. Since the angle change amplitude represents the degree of movement, the larger the angle change, the more intense the movement. The degree of exercise is greater than that of subject B, and subject A is used as the target subject during screening.

Of course, the screening rule can also be set according to other information. For example, different weights can be set for different categories of subjects in advance, and the subject with the highest weight among multiple subjects is taken as the target subject, or it can also be detected. When there are multiple subjects, prompt the user to select one of the specific subjects as the target subject.

S103, focus tracking on the target subject.

After the target subject is determined, when taking pictures or videos, the target subject is used as the focus target, and the focus of the target subject is continuously tracked. Optionally, if the target subject is not determined from the focused field of view, the subject is focused according to the central area of the focused field of view.

If the target subject is not determined, it means that no one in the focus field has a clear movement. For example, everyone is standing upright and taking pictures. In this case, the subject can be focused according to the central area of the focus field, and there is no need to target a subject. special focus.

In the subject focusing method provided in this embodiment, the posture information of each subject is obtained from the focusing field of view, and the special posture recognition or motion degree recognition made by each subject can be estimated through the subject posture information, and based on preset screening rules , to determine the target subject, so that the target subject can be effectively focused when taking pictures or recording videos. This method automatically focuses on the target subject instead of manual focus, which improves the flexibility and accuracy of focusing, while improving the experience of use. In addition, this method can improve the camera's focus function to capture the subject's posture and motion, so that the focus can more intelligently understand human behavior and meet user needs.

Generally, the focusing function of the camera needs to be used in both taking pictures and recording videos. For the two cases of taking pictures and recording videos, the target subject is determined from the focusing field of view. The corresponding embodiments are provided below for specific description.

In one embodiment, the above-mentioned screening rules include gesture action rules and/or exercise degree rules; the gesture action rules are used to indicate the priority of each gesture action; the movement degree rules are used to indicate the priority of each movement degree.

Among them, the gesture action rule includes the priority of a plurality of gesture actions, the gesture action rule is the rule for determining the target subject when taking pictures, and the movement degree includes the priority of each movement degree, which is used to determine the target subject when recording video. Rules, the correspondence between the degree of exercise and the priority can be determined according to the actual situation. For example, different degrees of exercise can be scored, and the higher the score is higher. Of course, the priority of the lower score can also be higher. This is not limited. Since photographing and capturing belong to static images, and video recording belongs to capturing dynamic images, two rules are used to determine the target subject in the corresponding situation, which can make the determination result of the target subject more accurate and the determination method more flexible.

As shown in FIG. 3 , an embodiment is provided. This embodiment is the process of determining the target subject from the focus field of view by the electronic device according to the subject posture information and the posture action rules, that is, the process of determining the focus target when taking pictures. Then step S102 includes:

S201 , according to the subject posture information, determine the posture action of each subject in the focused field of view.

At the beginning, the posture information of each subject in the focused field of view has been obtained, and according to the posture information of the subject, the posture action of each subject can be determined. For example, the specific posture or action of the subject is identified by focusing on the limb position of each subject in the field of view. As shown in Figure 3a, the posture action includes open arms with both hands, and tilted one leg.

S202, according to the gesture action rules, determine the subject with the highest priority in each gesture action as the target subject.

The gesture action rules include the priorities of multiple gesture actions, that is, different gesture actions correspond to different priorities. In this step, according to the gesture action rules, the subject of the gesture action with the highest priority is selected as the target subject, for example , if the priority of raising one leg is higher than the priority of opening both arms, it can be determined that in Figure 3a, the female dancer is the target subject.

In this embodiment, the subject of the gesture action with a high priority is determined as the target subject according to the gesture action rule. The gesture action rule may be a rule set according to the actual user's idea. The target subject determined in this way is relatively close to the user's needs. The target subject is focused during the photographing process of the scene, which improves the accuracy of focusing.

Optionally, an implementation manner of determining the gesture actions of each subject in the focused field of view is provided, as shown in FIG. 4 , the implementation manner includes:

S301, according to the subject posture information, obtain the angle between each subject's limbs.

The posture information of the subject includes the position information of the limbs of the subject, so that the angle between the limbs of the subject can be determined according to the position information of the limbs of the subject. Continuing to refer to FIG. 3a, by identifying the positions of the limbs of the female dancer and the male dancer in the figure, the included angles of the arms, legs and torso of the female dancer and the male dancer can be obtained.

S302, from the preset posture action information table, determine the posture action corresponding to each main body limb; the posture action information table includes the correspondence between the angles between the plurality of main body limbs and the posture action.

According to the acquired angle of each main body limb, the posture action corresponding to each main body limb is determined from the posture action information table. Among them, the posture action information table pre-sets the correspondence between the angles between the multiple main limbs and the posture actions. After the different angles before each limb are combined, they can correspond to different posture actions. For example, in Figure 3a, the female dance The angle before the dancer's legs and the angle between the legs and the torso can determine that the female dancer's one leg is in a tilted state at this time, that is, the corresponding posture action is a single-leg tilt.

Different angle values can define different posture actions. For example, the angle of the male dancer’s arms is larger than that of the female dancer. In this case, the arms can be set according to the range of the angle. The posture action with a large angle is to open the arms, and the posture action of the large angle of the arms open is to open the arms slightly. Certainly, the posture action can also be combined with the angles of all limbs to determine a specific posture, for example, the golden rooster is independent, the hands are raised, etc., and the setting of the posture action is not limited in this embodiment.

In this embodiment, for the angles between different limbs corresponding to a gesture action, the electronic device determines the gesture action according to the limb angle, which improves the accuracy of the determination of the gesture action. Determining the names of different gestures and actions makes the application more flexible and interesting to meet the needs of different users.

As shown in FIG. 5, an embodiment is provided. This embodiment is the process of determining the target subject from the focus field of view by the electronic device according to the subject posture information and motion degree rules, that is, the process of determining the focus target when recording video. Then step S102 includes:

S401 , according to the subject posture information, acquire the motion degree of each subject in the focused field of view in consecutive N frames.

To obtain the movement degree of each subject in consecutive N frames, the variation range of the position of each subject's limb can be determined according to the change of the posture information of the subject in each frame. The larger the variation range, the greater the movement degree.

S401 , according to the exercise degree rule, determine the subject with the highest priority in each exercise degree as the target subject.

The exercise degree rule includes priorities of multiple exercise degrees, that is, different exercise degrees correspond to different priorities.

In this step, according to the exercise degree, the subject corresponding to the exercise degree with the highest priority is determined as the target subject. In this way, according to the rules of the degree of motion, the subject of the gesture action with high priority is determined as the target subject, and the target subject is focused during the video recording process of the multi-person scene, which can automatically and continuously focus on the target subject required by the user. user experience.

In one embodiment, an implementation manner of determining the degree of movement of each subject in the focused field of view is provided, as shown in FIG. 6 , the implementation manner includes:

S501 , according to the main body posture information, obtain the angle change of each main body limb key point in N consecutive frames.

The subject posture information includes the position information of each subject limb, then the key points on each subject limb can be determined, and the angle change of each key point in consecutive N frames can be obtained, for example, the posture of the same person in multiple frames can be calculated. , to obtain the angle change of the key point of the same person between two frames (between any two consecutive frames).

S502 , from the preset motion degree information table, determine the motion degree corresponding to the angle change of each main body limb key point; the motion degree information table includes the corresponding relationship between the angle change amount and the motion degree of a plurality of main body limb key points.

According to the angle change of the key points of the subject's limbs, the corresponding motion degree is determined from the motion degree information table. Different angular changes of key points correspond to different degrees of motion. The degree of motion is represented by numerical quantification. The larger the angle change value, the higher the degree of motion.

The corresponding relationship between the angle changes of multiple key points of the subject's limbs and the degree of movement is set in advance in the movement degree information table, and the electronic device can obtain the movement degree of each subject in the focused field of view from the information table, ensuring the movement degree. Accuracy of acquisition.

Usually, in the photo application and video recording application, the target subject that the user focuses on is the person himself. The above two methods of determining the target subject according to the gesture action and determining the target subject according to the degree of movement involve the priority rule table and preset information. Tables can be set according to user needs, and rules that meet the needs can be set to determine the target subject. In this way, during the focusing process, people's behavior can be more intelligently understood, thereby enhancing the interest of users.

After the target subject is selected, the target subject is continuously tracked, but considering the scene of target subject replacement, an embodiment of replacing the target subject is provided. In one embodiment, if the target subject does not exist in the focused field of view, or , the duration of the focus tracking of the target subject reaches the preset duration, and a new target subject is re-determined from the focus field of view.

For example, when recording a video, the subject with the highest priority of motion is selected for focus tracking. Focus tracking means: once subject A is in focus, when the priority of subject A’s motion level is switched to non-first, the next k frames Keep the focus on subject A, and then see whether subject A returns to the first position at the kth frame. If so, continue to focus, and if not, perform focus switching. It is also possible to set a preset duration, to continuously focus on subject A within the preset duration, and to switch the focus target when the preset duration is reached; it is also possible to switch the focus if subject A disappears from the focus field of view.

In addition, the above-mentioned process of acquiring the subject posture information of each subject in the focused field of view provides an embodiment. As shown in FIG. 7 , the process of "obtaining the subject posture information of each subject in the focused field of view" includes:

S601: Detect key bone information of each subject in the focused field of view.

The electronic device detects the key bone information of each subject from the focused field of view. Taking the subject as a human as an example, the current deep learning algorithm with better effect can be used to detect the key points of multiple skeletons, such as openPose or open_pifpaf and other algorithms, as shown in Figure 7a Human skeleton keypoint information shown.

S602: Determine the posture information of each subject according to the key bone information.

In this embodiment, the posture information of each subject is determined according to the key bone information. For example, by using the deep learning method to detect the key point information of the human body of a single person or multiple people, the posture information of each subject in the focused field of view can be obtained effectively and accurately.

Illustratively, taking the subject as a human example, an embodiment of a subject focusing method corresponding to photographing (see FIG. 8 ) and an embodiment of a subject focusing method corresponding to video recording (see FIG. 9 ) are provided,

Wherein, the photographing and focusing steps include:

S11, the camera is activated;

S12, detect the key points of the human body from the focused field of view;

S13, output n human body posture information according to the key points of the human body;

S14, according to the human body posture information, perform human body posture action estimation;

S15, whether there is a human body b with the highest posture action priority; if so, go to S16, if not, go to S17;

In this step, if the judgment result is that there is no human body with the highest posture action priority, it means that the posture action priorities of all the people in the focus field are the same, for example, everyone is standing in one posture.

S16, the camera focuses normally;

S17, focus on human body b, and start to focus;

S18, follow the focus on the human body b: continue to focus on the human body b in the K frame;

S19, the subject is re-detected after K frames or after the disappearance of the human body b.

Wherein, the video recording and focusing steps include:

S21, the camera is activated;

S22, detect the key points of the human body from the focused field of view;

S23, output n human body posture information according to the key points of the human body;

S24, according to the human body posture information, accumulate N frames to calculate the human body motion degree, and perform priority sorting;

S25, whether there is a human body b with the highest exercise degree priority, if so, go to S26, if not, go to S27;

In this step, if the judgment result is that there is no human body with the highest priority of motion degree, it means that the priority of motion degree of all people in the focus field is the same, for example, everyone is standing in one posture.

S26, the camera focuses normally;

S27, focus on the human body b, and start to follow the focus;

S28, follow the focus on the human body b: continuously focus on the human body b in the K frame;

S29, the subject is detected again after K frames or after the human body b disappears.

The processes in the above-mentioned embodiments of FIG. 8 and FIG. 9 are the same as those involved in the previous embodiments, and are not repeated here. In the embodiments of Fig. 8 and Fig. 9, the human skeleton estimation is combined to solve the problem of portraits, especially in the scene of multiple portraits, paying attention to the person who has a specific movement, using the deep learning method to detect the human body key point information of a single person or multiple people, through the key point information The information predicts the recognition of special gestures or the degree of movement made by people, so as to guide the focusing function of photos or videos, and improve the camera focusing function to capture human posture and motion, so that the focusing can more intelligently understand human behavior and improve user experience,

It should be noted that, at the beginning of the implementation of this solution, before acquiring the posture information of each subject in the focused field of view, each subject needs to be detected from the focused field of view. In practical application, the subject detection model can be used for detection. Among them, the subject detection model is obtained by collecting a large amount of training data in advance, and inputting the training data into the subject detection model including the initial network weights for training.

It should be understood that although the steps in the flowcharts of FIGS. 2-9 are shown in sequence according to the arrows, these steps are not necessarily executed in the sequence shown by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in FIGS. 2-9 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed and completed at the same time, but may be executed at different times. These sub-steps or stages are not necessarily completed at the same time. The order of execution of the steps is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of sub-steps or stages of other steps.

In one embodiment, as shown in FIG. 10, a main body focusing device is provided, and the device includes: an acquisition module 10, a determination module 11 and a focus tracking module 12, wherein,

an acquisition module 10, configured to acquire subject posture information of the subject in the focused field of view;

The determination module 11 is used to determine the target subject from the focus field of view according to the subject posture information and preset screening rules; the screening rules are rules determined according to the gesture action and/or the degree of movement;

The focus tracking module 12 is used for tracking focus on the target subject.

In one embodiment, the above-mentioned screening rules include gesture action rules and/or motion degree rules;

Gesture action rules are used to indicate the priority of each gesture action;

The motion level rules are used to indicate the priority of each motion level.

In one embodiment, as shown in FIG. 11, a main body focusing device is provided, and the above determination module 11 includes: a first determination unit 111 and a second determination unit 112, wherein,

The first determining unit 111 is configured to determine the gesture action of each subject in the focused field of view according to the subject gesture information;

The second determining unit 112 is configured to determine the subject with the highest priority in each gesture action as the target subject according to the gesture action rule.

In one embodiment, the above-mentioned first determining unit 111 is specifically configured to obtain the angle between the limbs of each subject according to the posture information of the subject; from the preset posture action information table, determine the posture action corresponding to each limb of the subject; the posture The action information table includes the correspondence between the angles between the limbs of a plurality of subjects and the posture actions.

In one embodiment, the above-mentioned first determining unit 111 is further configured to obtain the motion degree of each subject in the focused field of view in consecutive N frames according to the subject posture information;

The above-mentioned second determining unit 112 is further configured to determine the subject with the highest priority in each exercise degree as the target subject according to the exercise degree rule.

In one embodiment, the above-mentioned first determining unit 111 is specifically configured to acquire, according to the main body posture information, the angular change amount of each main body limb key point in consecutive N frames; from the preset motion degree information table, determine each main body limb The motion degree corresponding to the angular change of the key point; the motion degree information table includes the correspondence between the angular change of the key points of the main body and the motion degree.

In one embodiment, the device includes a detection module for re-determining a new target subject from the focused field of view if the target subject does not exist in the focused field of view, or if the target subject's focus tracking duration reaches a preset duration.

In one embodiment, as shown in FIG. 12, a main body focusing device is provided, and the above-mentioned acquisition module 10 includes: a detection unit 101 and a posture unit 102,

A detection unit 101, configured to detect key bone information of each subject in the focused field of view;

The posture unit 102 is configured to determine the posture information of each subject according to the key bone information.

In one embodiment, the focus tracking module 12 is further configured to focus on the subject according to the center area of the focus field if the target subject is not determined from the focus field of view.

The implementation principles and technical effects of all the above-mentioned embodiments of the main body focusing apparatus are similar to those of the above-mentioned embodiments of the main body focusing method, which will not be repeated here.

For the specific limitation of the main body focusing device, reference may be made to the above definition of the main body focusing method, which will not be repeated here. Each module in the above-mentioned main body focusing device can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, the computer device may be a terminal, and its internal structure diagram may be as shown in FIG. 13 . The computer equipment includes a processor, memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a subject focusing method. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.

Those skilled in the art can understand that the structure shown in FIG. 13 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, a computer program is stored in the memory, and the processor implements the following steps when executing the computer program:

Obtain the subject posture information of the subject in the focused field of view;

According to the subject posture information and the preset screening rules, the target subject is determined from the focused field of view; the screening rules are rules determined according to the gesture action and/or the degree of movement;

Focus on the target subject.

The implementation principle and technical effect of the computer device provided by the above-mentioned embodiment are similar to those of the above-mentioned method embodiment, which will not be repeated here.

In one embodiment, a computer-readable storage medium is provided on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtain the subject posture information of the subject in the focused field of view;

According to the subject posture information and the preset screening rules, the target subject is determined from the focused field of view; the screening rules are rules determined according to the gesture action and/or the degree of movement;

Focus on the target subject.

The implementation principle and technical effect of the computer-readable storage medium provided by the above-mentioned embodiments are similar to those of the above-mentioned method embodiments, and details are not described herein again.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (12)

1. A subject focusing method, wherein the method comprises: Obtain the subject posture information of the subject in the focused field of view; Determine the target subject from the focused field of view according to the subject posture information and a preset screening rule; the screening rule is a rule determined according to the gesture action and/or the degree of movement; Focusing is performed on the target subject. 2. The method according to claim 1, wherein the screening rules comprise gesture action rules and/or motion degree rules; The gesture action rule is used to indicate the priority of each gesture action; The exercise degree rule is used to indicate the priority of each exercise degree. 3. The method according to claim 2, wherein determining the target subject from the focused field of view according to the subject posture information and a preset screening rule, comprising: According to the subject posture information, determine the posture action of each subject in the focused field of view; According to the gesture action rules, the subject with the highest priority in each gesture action is determined as the target subject. 4 . The method according to claim 3 , wherein determining the posture actions of each subject in the focused field of view according to the subject posture information comprises: 5 . According to the subject posture information, obtain the angle between each subject's limbs; From the preset posture action information table, determine the posture action corresponding to each main body limb; the posture action information table includes the correspondence between the angles between the plurality of main body limbs and the posture action. 5 . The method according to claim 2 , wherein determining the target subject from the focused field of view according to the subject posture information and preset screening rules, comprising: 6 . According to the subject posture information, obtain the motion degree of each subject in the focus field of view in consecutive N frames; According to the exercise degree rule, the subject with the highest priority in each exercise degree is determined as the target subject. 6 . The method according to claim 5 , wherein the acquiring, according to the subject posture information, the degree of movement of each subject in the focused field of view in consecutive N frames comprises: 6 . According to the subject posture information, obtain the angle change of each subject limb key point in the continuous N frames; From the preset motion degree information table, determine the motion degree corresponding to the angular variation of the key points of each subject's limbs; the motion degree information table includes the correspondence between the angular variation of the key points of the subject's limbs and the degree of motion relation. 7. The method according to any one of claims 1-6, wherein the method comprises: if the target subject does not exist in the focus field of view, or, if the target subject's focus tracking duration reaches For a preset time period, a new target subject is re-determined from the focusing field of view. 8. The method according to any one of claims 1-6, wherein the acquiring the subject posture information of the subject in the focused field of view comprises: Detecting key bone information of each subject in the focused field of view; According to the skeleton key information, each of the subject posture information is determined. 9. The method according to any one of claims 1-6, wherein the method further comprises: If the target subject is not determined from the focus field of view, subject focusing is performed according to the central area of the focus field of view. 10. A main body focusing device, wherein the device comprises: an acquisition module, used to acquire the subject posture information of the subject in the focused field of view; a determination module, configured to determine a target subject from the focus field of view according to the subject posture information and a preset screening rule; the screening rule is a rule determined according to the gesture action and/or the degree of movement; A focus tracking module, configured to perform focus tracking on the target subject. 11. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when the processor executes the computer program . 12. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images