JP2025103933A - Electronic device and control method thereof - Google Patents

Abstract

[Problem] An object of the present invention is to identify a person to whom posture information is to be notified even when there are multiple people for whom posture information has been acquired. [Solution] A system includes a posture information acquisition means capable of acquiring posture information for multiple subjects, a subject area acquisition means capable of acquiring subject areas occupied by the subjects for whom posture information can be acquired for the multiple people, a notification subject determination means for determining the subject with the largest subject area as the notification subject, a posture determination means for acquiring the posture of the notification subject as a posture determination result, and a notification means for notifying the posture based on the posture determination result. [Selected Figure] Figure 8

Description

The present invention relates to an electronic device and a control method thereof, and in particular to a technology for notifying a user of their posture.

There is a technology that determines whether a user's posture deviates from an ideal posture. Patent Document 1 discloses a method of determining and notifying whether the posture of the user is the target posture, regardless of the posture relationship with the user, by normalizing the posture information of the user according to the positional relationship with the user. Patent Document 2 discloses a method of limiting the number of subjects that can be detected when multiple people are photographed.

JP 2021-135337 A JP 2019-133436 A

The method of Patent Document 1 does not mention notifying multiple people about their postures from which posture information has been acquired. In addition, the method of Patent Document 2 can impose a maximum limit on the number of people who are notified about the posture of the photographed people, but cannot specify the people to whom the posture is to be notified.

The present invention aims to provide an electronic device that can identify a person whose posture information can be obtained by using the subject area and distance.

To achieve the above object, the electronic device of the present invention is characterized by having a posture information acquisition means capable of acquiring posture information of multiple subjects, a subject area acquisition means capable of acquiring a subject area occupied by the subjects for which the posture information can be acquired for the multiple people, a notification subject determination means for determining the subject occupying the largest area of the subject area as the notification subject, a posture determination means for acquiring the posture of the notification subject as a posture determination result, and a notification means for notifying the posture based on the posture determination result.

According to the present invention, even when there are multiple people whose posture information has been acquired, it is possible to identify the person whose posture is to be notified.

1 is a front perspective view of a notification robot as an example of a device to which the configuration of this embodiment can be applied. 1 is a side view of a notification robot as an example of a device to which the configuration of this embodiment can be applied. 1 is a rear perspective view of a notification robot as an example of a device to which the configuration of this embodiment can be applied. 11 is a top view illustrating the installation position during use of a notification robot as an example of a device to which the configuration of this embodiment can be applied. FIG. 1 is a block diagram showing an example of the configuration of a notification robot as an example of an apparatus to which the configuration of this embodiment can be applied. 11 is a flowchart of a notification process in the present embodiment. 4 is a flowchart of a reference attitude data acquisition process in the present embodiment. 11A and 11B are diagrams illustrating an example of the movement of a notification robot in the present embodiment. 1 is a diagram showing an example of the positional relationship between a user and a notification robot in this embodiment; 11A and 11B are diagrams illustrating an example of normalization of distance and angle of a posture in the present embodiment. 11A and 11B are diagrams illustrating an example of normalization regarding rotation of a posture in this embodiment. 1A to 1C are diagrams for explaining forward and backward leaning postures of a user in this embodiment. 11 is a diagram for explaining the position of a user's shoulder in this embodiment. FIG. 5 is a flowchart of a main subject determination process for identifying a main subject from a subject region in the first embodiment. 13 is a top view illustrating a case in which a person whose posture information can be obtained and a person whose posture information cannot be obtained are photographed while using the notification robot in this embodiment 1. FIG. FIG. 2 is a diagram showing a captured image in the first embodiment. 11 is a flowchart of a main subject determination process for identifying a main subject from distance information in the second embodiment. 13 is a top view illustrating the distance relationship between the notification robot 101 and a person when a person whose posture information can be obtained and a person whose posture information cannot be obtained are photographed while the notification robot is being used in this embodiment 2. FIG.

A preferred embodiment 1 of the present invention will now be described with reference to the drawings.

Figures 1a, 1b, and 1c are external views of a notification robot 101 as an example of an imaging control device of the present invention.

Figure 1a is a front perspective view, Figure 1b is a left side view, and Figure 1c is a rear perspective view.

In FIG. 1a, the exterior 105 of the notification robot 101 is made of a resin molded product with a roughly spherical lower portion.

The distance sensor 102 is provided on the upper front surface of the notification robot 101 and measures the distance from the notification robot 101 to any part of the user.

In FIG. 1b, the distance measurement sensor 102 is configured to have an elevation angle α so that it can capture an image of the upper body of the user when the notification robot 101 is placed on a desk and used. Here, α is preferably between 20° and 50°.

The light notification unit 103 is a semi-transparent part of the main body 101 that transmits light from the internal light source so that it can be seen by the user.

In FIG. 1c, the power button 104 is provided on the back of the main body 101, and the user can press it to turn the power of the notification robot 100 on and off.

Figure 1d is a top view illustrating the installation position of the notification robot 101 during use.

The notification robot 101 is approximately 6 cm in diameter and 8 cm in height, and can be placed anywhere on the desk 106. For example, if there is a PC (personal computer) monitor on the desk 106, it can be placed in front of the user and below the monitor (position P), or if a laptop PC is being used or if it cannot be placed directly in front, it can be placed diagonally forward (position Q or position R).

Figure 2 is a block diagram showing an example of the configuration of the notification robot 101 in this embodiment 1.

The lens 201 is a lens group consisting of multiple lenses, but for simplicity, only one lens is shown here. The light collected by the lens 201 is guided to the distance measurement sensor 102. The distance measurement sensor 102 is an element that converts an optical image into an electrical signal, and passes the converted electrical signal (data) to the system control unit 202. The data passed includes information on the distance from the distance measurement sensor 102 to the user 211. In addition, the timing at which the distance measurement sensor 102 converts the optical image into an electrical signal is controlled by the system control unit 202.

The non-volatile memory 203 is an electrically erasable and recordable memory, and may be, for example, a Flash-ROM. The non-volatile memory 203 stores constants, programs, and the like for the operation of the system control unit 202. The programs referred to here are programs for executing the various flowcharts described later in this embodiment 1.

The system control unit 202 is a control unit consisting of at least one processor or circuit, and controls the entire notification robot 101. By executing the program recorded in the non-volatile memory 203 described above, each process of the present embodiment 1 described below is realized. For example, a RAM is used for the system memory 204, and constants and variables for the operation of the system control unit 202, programs read from the non-volatile memory 203, etc. are deployed. The system control unit 202 detects a person based on the data obtained from the distance measurement sensor 102, and extracts distance information to characteristic points of the person, such as the face, right shoulder, left shoulder, and chest. In addition, the system control unit 202 performs calculations using the extracted distance information to determine the posture of the user 211. At this time, the extracted distance information and the determined posture are recorded in the non-volatile memory 203 as posture determination results.

The system timer 205 is a timing unit that measures the time used for various controls and the time of the built-in clock.

The power button 104 is an operating member that switches the power of the notification robot 101 ON and OFF. The power supply control unit 206 is composed of a DCDC converter, a switch circuit that switches the blocks to which electricity is applied, etc. The power supply control unit 206 controls the DCDC converter based on instructions from the system control unit 202, and supplies the required voltage for the required period to the appropriate location in the notification robot 101. The power supply unit 207 is composed of primary batteries such as alkaline batteries or lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, Li batteries, AC adapters, etc.

The communication I/F unit 208 is an interface that connects to external devices wirelessly or via a wired cable to send and receive data.

The light notification unit 103 is composed of a light emitting element such as an LED controlled by the system control unit 202. The system control unit 202 controls the light notification unit 103 based on the posture determination result and notifies the user 211 of the posture determination result. Alternatively, the system control unit 202 controls the light notification unit 103 to notify the user 211 that the reference posture of the user 211 is being acquired (calibration is in progress).

The vibration unit 209 is composed of an actuator or the like, and is a mechanism that physically vibrates the notification robot 101, and is controlled by the vibration control unit 210.

The vibration control unit 210 is composed of a motor driver and the like, and is controlled by the system control unit 202. The vibration control unit 210 controls the presence or absence, timing, and amount of electricity supplied to the vibration unit 209, making it possible to vibrate the notification robot 101 with any amplitude and at any period.

The above is an explanation of the block diagram in the embodiment of the present invention, but it is not limited to this.

Next, the notification process of the notification robot 101 in this embodiment 1 will be explained using the flowcharts in Figures 3 and 4. This process is realized by the system control unit 202 expanding a program recorded in the non-volatile memory 203 into the system memory 204 and executing it. This process begins when the power button 104 of the notification robot 101 is pressed to turn on the power and the distance measurement sensor 102 becomes ready to start, causing the distance measurement sensor to start taking images.

In S301, the system control unit 202 starts capturing images using the distance measurement sensor 102.

In S302, the system control unit 202 determines whether or not the reference posture data is recorded in the non-volatile memory 203. If the reference posture data is recorded, the process proceeds to S304; if not, the process proceeds to S303.

In S303, the system control unit 202 acquires reference posture data. The acquisition of the reference posture data will be described later with reference to the flowchart of the reference data acquisition process in FIG. 4. Here, the reference posture data is the posture taken by the user after notifying the user to adopt a good posture, and in the process described later, it is determined whether the current posture of the user is good or not based on the reference posture data. By acquiring the reference posture data, if the user has a natural tendency to round shoulders, it is possible to determine whether the original tendency has led to round shoulders. If the reference posture data is not acquired from the target user and the reference posture data is used to determine whether the posture is good or bad, it is possible to reduce the possibility that the notification described later will be issued frequently when the user has a natural tendency or when the user is in a posture that is easy for the user to work in.

In S304, the system control unit 202 acquires a captured image using the distance measurement sensor 102.

In S305, the system control unit 202 detects a person from the captured image acquired in the previous S304.

In S306, the system control unit 202 acquires the distances to the head, shoulders, and chest of the subject detected in S305. The distances to the head, shoulders, and chest of the subject are described with reference to Figs. 6a to 6c. Figs. 6a to 6c are top views of the notification robot 101 and the subject 600. Note that the actual position of the notification robot 101 in Fig. 6a is 101_01, and its position when virtually facing forward is 101_02. The head distance is the distance 614 from the notification robot 101_01 to the subject's head 603. The shoulder distance is the distance 610 from the notification robot 101_01 to the left shoulder 605, and the distance 611 from the notification robot 101_01 to the right shoulder 606. The chest distance is the distance 615 from the notification robot 101_01 to the chest 604.

S307 is a sub-flow of the main subject determination process for determining the person whose posture is to be notified from the representative subject area in this embodiment 1, and will be described later after explaining the basic flow.

In S327, the system control unit 202 determines whether or not the main subject was identified in S307. If it is determined that the main subject was identified, the process proceeds to S308; otherwise, the process repeats S304 to S327.

In S308, the system control unit 202 normalizes the distances to the subject's face, shoulders, and chest acquired in S306 to a posture in which the notification robot 101 is directly in front of the user's face. The data is converted to data when the notification robot 101 is placed at a specific position perpendicular to the line connecting the shoulders 605, 606. The specific position is 101_02 in Figure 6a. The head distance is the distance 616 from the position of the virtual notification robot 101_01 to the subject's head 603. The head distance is converted to the distance 616 from the position of the virtual notification robot 101_01 to the subject's head 603.

Here, we will use Figure 6b to explain the specific conversion method for normalization. It is necessary to perform corrections for distance and angle, as well as corrections for the rotation of the subject. First, we will explain the corrections for distance.

The virtual subject 630 in Fig. 6b is an image of the subject 600 captured by the virtual notification robot 101_02 in Fig. 6a, but without correction of the left shoulder 605 and the right shoulder 606. The virtual notification robot 101_03 in Fig. 6b is a device in which the notification robot 101 in Fig. 6a and the virtual notification robot 101_02 in Fig. 6a are virtually superimposed.

6b, a distance 614 to the head 603 of the subject 600 and a distance 616 to the head 633 of the subject 630 form an angle 631. The direction in which the head 633 of the subject 630 and the imaging plane of the virtual notification robot 101_03 intersect at right angles is defined as the z-axis, and the direction perpendicular to the z-axis is defined as the x-axis. If the position of the virtual notification robot 101_03 is defined as the origin, and the z coordinate of the left shoulder 606 is defined as coordinate 606_z and the x coordinate is defined as coordinate 606_x, then the z coordinate 636_z and the x coordinate 636_x of the left shoulder 636 of the virtual subject 630 are expressed as follows:
(A) Coordinate 636_z = (distance 616 / distance 614) × coordinate 606_z × cos (angle 631) - (distance 616 / distance 614) × coordinate 606_x × sin (angle 631),
(B) Coordinate 636_x = (distance 616 / distance 614) × coordinate 606_z × sin (angle 631) + (distance 616 / distance 614) × coordinate 606_x × cos (angle 631),
It can be found by:

Similarly, in FIG. 6b, assuming that the position of the virtual notifying robot 101_03 is the origin, the z coordinate of the right shoulder 605 is the coordinate 605_z, and the x coordinate is the coordinate 605_x, the z coordinate 635_z and the x coordinate 635_x of the left shoulder 635 of the virtual subject 630 are expressed as follows:
(C) Coordinate 635_z = (distance 616 / distance 614) × coordinate 605_z × cos (angle 631) - (distance 616 / distance 614) × coordinate 605_x × sin (angle 631),
(D) Coordinate 635_x = (distance 616 / distance 614) × coordinate 605_z × sin (angle 631) + (distance 616 / distance 614) × coordinate 605_x × cos (angle 631),
It can be found by:

Next, we will use Figure 6c to explain how to correct for subject rotation.

Virtual subject 650 is virtual subject 630 after rotation correction has been performed.

The line segment connecting the left shoulder 655 and the right shoulder 656 of the virtual subject 650 and the line segment connecting the left shoulder 635 and the right shoulder 636 of the virtual subject 630 have an angle 651. If the position of the virtual notification robot 101_03 is taken as the origin, the z coordinate 656_z and the x coordinate 656_x of the left shoulder 656 of the virtual subject 650 are expressed as follows:
(E) Coordinate 656_z = Coordinate 636_z × cos (angle 651) - (coordinate 636_x - distance 616) × sin (angle 651),
(F) Coordinate 656_x = Coordinate 636_z × sin (angle 651) + (coordinate 636_x - distance 616) × cos (angle 651) + distance 616,
It can be found by:

Similarly, when the position of the virtual notifying robot 101_03 in FIG. 6c is taken as the origin, the z-coordinate 655_z and the x-coordinate 656_x of the left shoulder 655 of the virtual subject 650 are expressed as follows:
(G) Coordinate 655_z = Coordinate 636_z × cos (angle 651) - (coordinate 635_x - distance 616) × sin (angle 651),
(H) Coordinate 655_x = Coordinate 636_z × sin (angle 651) + (coordinate 635_x - distance 616) × cos (angle 651) + distance 616,
It can be found by:

The order of the distance and angle corrections and rotation corrections mentioned above does not matter. Also, the above example is a two-dimensional correction when viewed from above, and the correction is made using a two-dimensional rotation matrix and distance. If correction is required for the subject's height, correction is required using a three-dimensional rotation matrix and distance that adds a height correction to the above formula.

In S309, the system control unit 202 calculates the difference value Zhn between the head and chest, and the difference value Zcn between the midpoint of the line connecting both shoulders and the chest based on the values normalized in S308, and records these in the system memory 204.

As shown in Figure 7a, the difference value Zhs between the head 711 and chest 712 in the reference posture, and an example of a posture when performing notification processing is the difference value Zhn between the head 713 and chest 714. On the coordinate axis 701, the direction away from the notification robot 101 is positive. If Zhn is smaller than Zhs (negative value), that is, when the head is larger than the chest and comes forward, it is determined to be a forward leaning posture. If Zhn is larger than Zhs, it is determined to be a backward leaning posture (loose hips). As shown in Figure 7b, the difference value Zcs between the midpoint 715 of the line connecting both shoulders in the reference posture and the chest 716, and an example of a posture when performing notification processing is the difference value Zcn between the midpoint 717 of the line connecting both shoulders and the chest 718. If Zcn is smaller than Zcs (negative value), it is determined to be a hunched shoulder posture.

In S310, the system control unit 202 compares the difference value Zhs between the head and chest in the reference posture acquired in S303, and the difference value Zcs between the midpoint of the line connecting both shoulders and the chest with Zhn and Zcn acquired in S309, respectively. In other words, Zhs is compared with Zhn, and Zcs is compared with Zcn.

In S311, the system control unit 202 determines whether either (I) Zhn-Zhs<x1 or (II) Zhn-Zhs>x2 holds, or neither holds. That is, in (I), it is possible to determine whether the posture is leaning forward. Note that x1 is, for example, a value such as -6 cm, -8 cm, or -10 cm. In (II), it is possible to determine whether the posture is leaning backward (loosening of the legs). Note that x2 is, for example, a value such as 2 cm, 4 cm, or 6 cm.

If it is determined that either (I) or (II) is true, the process proceeds to S312; otherwise, the process proceeds to S313. Note that x1 and x2 may be arbitrarily set by the user, or may be changed automatically based on the input user profile (age, sex, build). For example, when shoulder width is detected, if the shoulder width is large, x1 and x2 are changed to larger values. Alternatively, if the sex is female, x1 and x2 are changed to smaller values.

In S312, the system control unit 202 starts timing the forward/backward tilt posture count T1. T1 indicates the time during which the user's posture continues to be leaning forward more than the reference posture, or in a state of being in a relaxed state. Note that if timing of T1 has already started, timing continues in S312.

In S313, the system control unit 202 determines whether the count T1 of the forward/backward tilt posture has elapsed for 5 minutes or more. If it is determined that T1 has elapsed for 5 minutes or more, the process proceeds to S314, and if not, the process proceeds to S316.

In S314, the system control unit 202 notifies the user that the posture is tilted forward or backward. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with any amplitude and any period. Note that the notification method may be different depending on whether the posture is leaning forward or the posture is limp.

In S315, the system control unit 202 resets the forward/backward tilt posture time T1 to 0. In other words, if the user was temporarily leaning forward or losing his balance, but returns to a posture close to the reference posture before five minutes have elapsed, T1 is reset.

In S316, the system control unit 202 determines whether or not (III) Zcn-Zcs<x3. That is, in (III), it is possible to determine whether or not the user has rounded shoulders. Note that x3 is, for example, a value such as -1.0 cm, -1.5 cm, or -2.0 cm. x3 may be set arbitrarily by the user, or may be changed automatically based on the input user profile (age, sex, build). For example, the shoulder width is detected, and if the shoulder width is large, x3 is changed to a larger value. Alternatively, if the gender is female, x3 is changed to a smaller value.

As described above, since Zhn, Zhs, Zcn, and Zcs all use the z-coordinate of the chest, it is necessary for the notification robot 101 to detect the position of the user's chest. Therefore, it is desirable that angle 651 in FIG. 6c, that is, the angle between the z-axis plane of the notification robot 101 and the left and right shoulders of the subject user, be in the range of approximately -80° to +80°. Within this range, the notification robot 101 can be placed at any position relative to the user.

If it is determined that (III) is true, proceed to S319; otherwise, proceed to S321.

In S317, the system control unit 202 starts counting T2 of the rounded shoulder posture. T2 indicates the time during which the user's posture continues to be more rounded than the reference posture. Note that if the counting of T2 has already started, the counting continues in S317.

In S318, the system control unit 202 determines whether the rounded shoulder posture count T2 has elapsed for 5 minutes or more. If it is determined that T2 has elapsed for 5 minutes or more, the process proceeds to S319, and if not, the process proceeds to S321.

In S319, the system control unit 202 notifies the user that the user's posture is rounded shoulders. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with any amplitude and any period.

In S320, the system control unit 202 resets the rounded shoulder posture time T2 to 0. In other words, if the user was in a rounded shoulder state temporarily but returned to a posture close to the reference posture before five minutes had elapsed, T2 is reset.

In S321, the system control unit 202 determines whether the user has stood up. If it is determined that the user has stood up, the process proceeds to S322; otherwise, the process proceeds to S323.

In S322, the system control unit 202 resets the seating time T3, which will be described later.

In S323, the system control unit 202 starts or continues to measure the seating time T3. The seating time T3 is a time for detecting that the user is continuously sitting. Note that the timing of T1, T2, and T3 is performed by the system timer 205.

In S324, the system control unit 202 determines whether the seated time T3 is 30 minutes or more. If it is determined that T3 is 30 minutes or more, the process proceeds to S325; otherwise, the process proceeds to S326.

In S325, the system control unit 202 notifies the user that they have been sitting too long. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with any amplitude and at any period.

Here, various notification methods of the notification robot 101 will be explained using Figure 5.

In FIG. 5, the notification robot 101 is in the following states: A (normal position), B (swaying left), and C (swaying right).

In notification pattern 1, the state transition is as follows: state A → B → C → A → stopped for 1 second → B → C → A → stopped. By swinging left and right, the user can easily check the notification even when the notification robot 101 is placed on a desk.

In notification pattern 2, the state transitions are A → B → C → B → C → B → C → A → Stopped.

In notification pattern 3, the light notification unit 103 flashes twice at 2 Hz while remaining in state A.

In notification pattern 4, the light notification unit 103 lights up once for 2 seconds while remaining in state A.

In notification pattern 5, the light notification unit 103 blinks three times at 1 Hz while remaining in state A.

In notification pattern 6, state A remains and the light notification unit 103 lights up brightly once for one second.

Notification pattern 1 is used in the notification of poor posture S314 and the notification of rounded shoulders S319 in FIG. 3. Notification pattern 2 is used in the notification of sitting too long S325.

Notification pattern 3 is used as a notification when the power button 104 is pressed and the notification robot 101 starts taking pictures (S301).

Notification pattern 4 is used as a notification when a person is detected (S305). This is also used when a person is detected again after the user has stood up during the sitting too long detection S325 (S321).

In S326, the system control unit 202 determines whether the power of the notification robot 101 has been turned off. If it is determined that the power has been turned off, the process of FIG. 3 ends, and if not, the process returns to S304. Note that S326 may determine Yes when the power of the PC (personal computer) used by the user is turned off, or may determine Yes when a predetermined time has arrived.

Next, the flowchart for acquiring reference posture data will be explained using FIG. 4. Note that this process starts when the process proceeds to S303 in FIG. 3.

In S401, the system control unit 202 controls the light notification unit 103 to notify the user that the reference posture will be taken. At this time, the notification is made using notification pattern 5.

In S402, the system waits for a signal indicating that the user is ready. The system determines that the user is ready when the power button 104 is pressed and held down. The user's signal of OK may be a gesture such as the user raising both hands.

In S403, the system control unit 202 acquires the distances between the head, shoulders, and chest as the reference posture of the detected subject. At this time, notification pattern 6 is used to notify the successful acquisition of the reference posture.

In S404, the system control unit 202 normalizes the distances between the subject's reference face, shoulders, and chest obtained in S403 to a posture that is taken when the notification robot 101 is directly in front of the user's face.

In S405, the system control unit 202 calculates the difference value Zhs between the line connecting both shoulders and the head, and Zcs between the line connecting both shoulders and the chest based on the values normalized in S404, and records these in the volatile memory 203.

Next, the method for determining the person whose posture is to be notified using the subject area when multiple people are photographed in the main subject determination process of S307 will be explained using the flowchart in FIG. 8.

In S801, the system control unit 202 acquires posture information of a person in a captured image. In this embodiment 1, the posture that can be calculated from the face and both shoulders in the captured image is acquired as posture information. In this embodiment, the coordinates that correspond to the face and both shoulders detected in S305 are acquired as posture information.

In S802, the system control unit 202 determines whether or not a person whose posture information can be obtained is captured in the captured image acquired in S305. If it is determined that a person whose posture information can be obtained is captured in the image, the process proceeds to S803; otherwise, the process ends as it is not possible to make a determination.

Here, we will explain people for whom posture information can be obtained and people for whom posture information cannot be obtained, using Figures 9a and 9b.

In the first embodiment, as described above, a person whose posture information can be obtained is defined as a person whose face and both shoulders are included in the captured image and whose posture can be detected, and whose posture cannot be detected if both shoulders are not included.

Figure 9a shows an example in which the main subject 901, who is the user, other subjects 902 and 903, a subject 904 whose posture information cannot be obtained, the left shoulder 905 and right shoulder 906 of the main subject 901, and the left shoulders 907, 909 and right shoulders 908, 910 of the subjects 902 and 903 are photographed while the notification robot 101 is in operation.

Figure 9b shows an image captured by the notification robot 101, and indicates the correspondence with Figure 9a 901-910, as well as the differences 911-913 in the x-coordinates of both shoulders of each person.

In S803, the system control unit 202 acquires the above-mentioned x-coordinate differences 911 to 913. In this example, the magnitude relationship of the x-coordinate differences is 911>912>913.

In S804, the system control unit 202 determines the subject area as shoulder width and determines the largest person from the shoulder width. In the example of Figures 9a and 9b, the main subject 901 with the largest difference in the x coordinates of both shoulders is determined as the person to be notified of the posture. Note that when posture information can be obtained for only one person, that person is determined as the main subject.

According to the first embodiment described above, even when multiple people are photographed and posture information for each person has been acquired, the person whose posture is to be notified can be identified by determining the person with the largest subject area.

In this embodiment, the person to be notified of the posture is determined based on the difference value of the x coordinate only, but the present invention is not limited to this, and the size may be calculated based on the x and y coordinates.

In this embodiment 1, the person to be notified of the posture is determined by both shoulders, but this is not limited to the above. The coordinate difference between the positions of the face and shoulders or the face and chest may be calculated as the subject area and then compared.

In this embodiment, the description is based on the assumption that the subjects are facing forward, but the difference in x-coordinates may be normalized according to the orientation of each subject, and the largest person may be the subject of notification.

Next, we will explain the second embodiment of the present invention.

In this second embodiment, a method is described for the notification robot 101 to determine the person to be notified of the posture using distance information when multiple people are photographed. Note that in the second embodiment, the method for identifying the person to be notified of the posture when posture information of multiple people is acquired is different from that in the first embodiment. That is, while FIG. 3 is common, there is a difference in the process of determining the main subject in S307, and the difference from the first embodiment is described using the flowchart in FIG. 10.

In FIG. 10, in the main subject determination process of S307, the process flow has been changed to S1001 and S1002 from S803 and S804 in FIG. 8 of the first embodiment, and since steps S801 and S802 are common to the first embodiment, only the differences from FIG. 8 will be explained.

In S1001, the system control unit 202 acquires the average distance between the shoulders of a person for whom posture information can be acquired.

Here, we will use Figure 11 to explain how to obtain the distance of a person from which posture information can be obtained.

Figure 11 shows 901 to 910 in Figures 9a and 9b described above, as well as distance 1101 of main subject 901, and distances 1102 and 1103 of subjects 902 and 903. The method for obtaining each of the distances 1101 to 1103 is to obtain the average value of the distances between both shoulders. In this embodiment 2, the distances are 1101>1102>1103.

In S1002, the system control unit 202 determines the person whose average shoulder distance obtained in S1001 is closest to the notification robot 101. Here, the person who is the main subject 901 and whose average shoulder distance is closest is set as the posture notification target. Note that when posture information can be obtained for only one person, that person is determined to be the main subject.

According to the second embodiment described above, even if multiple people whose posture information can be obtained are photographed, the person whose posture is to be notified can be identified by determining the person who is closest in distance.

However, this is not limited to this, and the distance between the left or right shoulder, and the distance of a part other than the shoulder that is necessary to obtain posture information may also be used.

Note that if multiple people whose posture information can be obtained are captured in a photograph, the system control unit 202 may not determine the notification target and may not issue a notification.

In addition, during a period in which multiple people whose posture information can be obtained are photographed, the system control unit 202 may not be able to distinguish the posture information, and the notification robot 101 may, for example, provide a different type of notification other than posture notification, such as by vibrating.

When multiple people whose posture information can be obtained are captured in an image, the notification target may be determined, for example, by recording the facial information of the individuals in the non-volatile memory 203 in S303, and by having the system control unit 102 identify the individuals from the faces in the captured image in S804 and S1002. For example, if only one individual can be identified, the identified individual may be the notification target.

Although it has been described in S314, S319, and S325 that the notification robot 101 notifies the user, the notification may also be sent to a compatible app in the user's smartphone or PC using the communication I/F unit 208. Furthermore, information about the user's posture may be recorded in the app and displayed in graphs and numerical values to make it easier for the user to understand the user's posture tendencies and habits. For example, if the user's working days are Monday through Friday, if the user notices that there are more notifications of a leaning back posture on Fridays, the user can consciously correct their posture.

In the above explanation, the notification method is the movement or light of the notification robot 101 itself, but in the case of a smartphone, a push notification may be sent by flashing the display or issuing a warning.

In addition, the notification robot 101 in this embodiment detects its posture based on the distance sensor 102, but it may also detect its posture using a visible light imaging sensor that does not measure distance. In that case, distance information is estimated from the captured image obtained by the imaging sensor.

The various controls described above as being performed by the system control unit 101 may be performed by a single piece of hardware, or multiple pieces of hardware may share the processing to control the entire device.

Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

In the above-mentioned embodiment, the present invention has been described as being applied to a notification robot 101, but the present invention is not limited to this example and can be applied to any electronic device that can obtain information about the positional relationship and posture of the device relative to the user. In other words, the present invention can be applied to mobile phone terminals, portable image viewers, printer devices with viewfinders, digital photo frames, music players, game consoles, e-book readers, etc.

Other Embodiments
The present invention can also be realized by executing the following process. That is, software (programs) that realize the functions of the above-mentioned embodiments are supplied to a system or device via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program codes. In this case, the program and the recording media on which the program is stored constitute the present invention.

101 Notification robot 102 Distance measurement sensor 103 Light notification unit 104 Power button 106 Desk 201 Lens 202 System control unit 203 Non-volatile memory 204 System memory 205 System timer

Claims (5)

A posture information acquisition means for acquiring posture information of a plurality of subjects,
a subject area acquisition means for acquiring a subject area occupied by a subject for which the posture information can be acquired for a plurality of persons;
a notification target determining means for determining, as a notification target, a subject that occupies the largest area of the subject area;
a posture determination means for obtaining the posture of the notification target as a posture determination result;
The electronic device further comprises a notification means for notifying the user of the posture based on the posture determination result. A posture information acquisition means for acquiring posture information of a plurality of subjects,
a subject distance acquisition means for acquiring a subject for which the posture information can be acquired and a distance between the electronic device and the subject for a plurality of people;
a notification target determining means for determining a subject that is closest to the electronic device as a notification target;
a posture determination means for obtaining the posture of the notification target as a posture determination result;
The electronic device further comprises a notification means for notifying the user of the posture based on the posture determination result. The notification target determination means
A personal authentication means for identifying the faces of a plurality of subjects,
3. The electronic device according to claim 1, wherein, when the posture information acquisition means is capable of acquiring posture information of a plurality of subjects, a notification target is determined based on an output from the personal authentication means. The electronic device according to any one of claims 1 and 2, characterized in that the predetermined notification is given by shaking the electronic device. The subject distance acquisition means is capable of acquiring subject information acquired by a distance measuring sensor,
whether the distance measuring sensor is at a first position in front of the subject or at a second position shifted from the first position, the control means normalizes the posture information of the subject based on the positional relationship acquired by the acquisition means;
3. The electronic device according to claim 2, further comprising: a control unit configured to control not to issue the specified notification when the normalized posture information of the subject satisfies the specified condition, and to control not to issue the specified notification when the normalized posture information of the subject does not correspond to the specified posture.