JP5935779B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for specifying an image related to a predetermined time point.

  2. Description of the Related Art Conventionally, an apparatus that identifies an image at the time of impact from moving image data obtained by capturing a subject performing a series of swing operations is known (see, for example, Patent Document 1). Specifically, the apparatus measures the ball speed by a Doppler sensor while storing moving image data in a ring buffer, and identifies a frame image at the time of impact based on the processing time taken for measuring the ball speed. .

JP 2011-30669 A

  However, in the case of the above-mentioned Patent Document 1, a special device such as a Doppler sensor is required to specify an image at a predetermined moment during a series of swing operations in moving image data.

  The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus, an image processing method, and a program capable of specifying a desired image without using a special device. It is to be.

In order to solve the above problems, an image processing apparatus according to the present invention provides:
An acquisition unit that acquires a plurality of images in which the motion of the subject is continuously captured, and a plurality of images that are smaller than the entire area of the image and that do not overlap each of the plurality of images acquired by the acquisition unit In the operation of the subject among the plurality of images, an area setting unit for setting a partial area and a variation in pixel values between the plurality of images in each of the plurality of partial areas set by the area setting unit And a specifying unit that specifies a plurality of images related to a predetermined time point.

The image processing method according to the present invention includes:
An image processing method using an image processing apparatus, wherein a process of acquiring a plurality of images obtained by continuously capturing the motion of a subject, and a range of each of the acquired plurality of images is larger than the entire area of the image Of the subject among the plurality of images based on a process of setting a plurality of partial regions that are small and non- overlapping and fluctuations in pixel values between the plurality of images in each of the plurality of partial regions that are set And a process of specifying a plurality of images related to a predetermined time point in the operation.

The program according to the present invention is
The computer of the image processing apparatus, acquiring means for acquiring a plurality of images operation of the subject is continuously captured in each of the plurality of images acquired by the acquisition unit, smaller range than the entire area of the image And a region setting means for setting a plurality of partial regions that do not overlap , and based on fluctuations in pixel values between the plurality of images in each of the plurality of partial regions set by the region setting means, It is characterized by functioning as specifying means for specifying a plurality of images related to a predetermined time point in the movement of the subject.

  According to the present invention, a desired image can be specified without using a special device.

It is a block diagram which shows schematic structure of the imaging device of one Embodiment to which this invention is applied. 3 is a flowchart illustrating an example of an operation related to an imaging process by the imaging apparatus of FIG. 1. 3 is a flowchart illustrating an example of an operation related to an image specifying process in the imaging process of FIG. 2. It is a figure which shows typically an example of the live view image which concerns on the imaging process of FIG. It is a figure which shows typically an example of the moving image which concerns on the imaging process of FIG. It is a figure which shows typically an example of the frame image which concerns on the image specific process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment to which the present invention is applied.
As illustrated in FIG. 1, the imaging apparatus 100 according to the present embodiment includes a central control unit 1, a memory 2, an imaging unit 3, an image data generation unit 4, an image processing unit 5, an image recording unit 6, A display unit 7 and an operation input unit 8 are provided.
The central control unit 1, the memory 2, the imaging unit 3, the image data generation unit 4, the image processing unit 5, the image recording unit 6, and the display unit 7 are connected via a bus line 9.

  The central control unit 1 controls each unit of the imaging device 100. Specifically, although not shown, the central control unit 1 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and includes various processing programs (illustrated) for the imaging apparatus 100. Various control operations are performed according to (omitted).

  The memory 2 is configured by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by each unit such as the central control unit 1 and the image processing unit 5.

  The imaging unit 3 images a subject (eg, a club head C of a golf club; see FIG. 4) that exists in the background. Specifically, the imaging unit 3 includes a lens unit 3a, an electronic imaging unit 3b, and an imaging control unit 3c.

The lens unit 3a includes a plurality of lenses such as a zoom lens and a focus lens, for example.
The electronic imaging unit 3b is composed of, for example, an image sensor such as a complementary metal-oxide semiconductor (CMOS) or a charge coupled device (CCD), and converts an optical image that has passed through various lenses of the lens unit 3a into a two-dimensional image signal. To do.
In addition, although illustration is abbreviate | omitted, the imaging part 3 may be provided with the aperture_diaphragm | restriction which adjusts the quantity of the light which passes the lens part 3a.

  The imaging control unit 3c controls imaging of the subject by the imaging unit 3. In other words, the imaging control unit 3c includes a timing generator, a driver, and the like, although not illustrated. Then, the imaging control unit 3c scans and drives the electronic imaging unit 3b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 3b every predetermined period, and the electronic imaging unit 3b. The frame image is read from the imaging area for each screen and is output to the image data generation unit 4.

The image data generation unit 4 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the frame image transferred from the electronic imaging unit 3b, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data). Further, the image data generation unit 4 encodes each of a plurality of frame images constituting the moving image in a predetermined format (for example, MPEG format, motion JPEG format, etc.), and generates moving image image data.
In addition, the image data generation unit 4 transfers the generated image data to the memory 2 used as a buffer memory.

The image processing unit 5 includes an image acquisition unit 5a, a region setting unit 5b, a decoding unit 5c, a threshold setting unit 5d, a frame number specifying unit 5e, and an image specifying unit 5f.
Note that each unit of the image processing unit 5 includes, for example, a predetermined logic circuit, but the configuration is an example and the present invention is not limited thereto.

The image acquisition unit 5a acquires an image to be processed by the image specifying process.
That is, the image acquisition unit (acquisition unit) 5a acquires a plurality of frame images F1 to F5 (see FIGS. 5A to 5E) in which the motion of the subject is continuously captured. Specifically, for example, the image acquisition unit 5 a captures a series of motions of a predetermined sport by a subject (for example, the motion of the club head C in a golf swing) by the image capturing unit 3 and the image data generation unit 4. Image data of a moving image composed of a plurality of generated frame images is acquired as a processing target of image specifying processing.
For example, the imaging unit 3 captures a series of operations of a predetermined sport by a subject at a predetermined imaging frame rate (for example, 1000 fps), and the image data generation unit 4 includes a plurality of frame images associated with frame numbers. Is generated and transferred to the memory 2. Here, when the subject is imaged by the imaging unit 3, the display unit 7 superimposes an index M for aligning the initial positions of the club head C and the ball B of the golf club on the live view image L, for example, OSD ( on-screen display) (see FIG. 4).
Then, the image acquisition unit 5a reads out the image data of the moving image stored in the memory 2 and acquires it as a processing target of the image specifying process.
Note that the image acquisition unit 5a may read out the image data of the moving image recorded in the image recording unit 6 after the imaging of the subject by the imaging unit 3 and acquire it as a processing target of the image specifying process.

The area setting unit 5b sets a subject detection area.
That is, the area setting unit (area setting unit) 5b is a time point when a subject (for example, a club head C) exists at a predetermined position for each of a plurality of frame images constituting the moving image acquired by the image acquisition unit 5a. A plurality of detection areas for specifying the frame number (identification information) is set. Specifically, the area setting unit 5b sets two substantially rectangular detection areas, for example, at positions near the both ends in the left-right direction of the substantially central part in the up-down direction for each of the plurality of frame images (FIG. 6A). ) To FIG. 6 (c)). Here, the position of the detection area of the subject is set with reference to the moving direction of the subject in a series of movements of the subject, for example. That is, for example, the area setting unit 5b sets at least two detection areas A1 and A2 at positions near the both ends of each frame image for a subject moving in the left-right direction, On the other hand, at least two detection areas (not shown) are set at positions near both ends in the vertical direction of each frame image.
In addition, the region setting unit 5b sets the subject detection region at a predetermined position of the first frame image F1 (see FIG. 6A) among the plurality of frame images, and similarly for the remaining frame images. The detection area is applied and set at a position corresponding to the detection area of the first frame image F1.
Note that the arrangement, number, shape, and the like of the detection regions described above are merely examples, and are not limited thereto, and can be arbitrarily changed as appropriate.

The decoding unit 5c decodes moving image image data including a plurality of frame images.
Specifically, for example, the decoding unit 5c applies the moving image to the moving image image data encoded in a predetermined format (for example, MPEG format or motion JPEG format) acquired by the image acquisition unit 5a. A decoding process according to the image data encoding method is performed to generate YUV data of a plurality of frame images.
At this time, the decoding unit 5c may perform a process of reducing each frame image to a predetermined size (for example, VGA or QVGA size) based on, for example, the display resolution of the display panel 7a.

The threshold setting unit 5d sets a threshold for detecting a subject.
That is, the threshold setting unit (threshold setting means) 5d sets the representative value of the pixel value of the detection region in a predetermined number of frame images within a predetermined period among the plurality of frame images as a threshold for detecting the subject. Specifically, for example, the threshold setting unit 5d is adjacent to each detection region for a predetermined number (for example, n; n is a natural number) of frame images from the first frame image F1 decoded by the decoding unit 5c. The sum of squared differences (SSD) of pixel values (for example, luminance values) from the frame image is calculated. Then, the threshold value setting unit (calculation unit) 5d calculates a representative value (for example, an average value, a median value, etc.) of a plurality of (for example, n−1) difference square sums SSD calculated for each detection region. Thus, it is set as a threshold value for specifying the frame number when the subject is present in the detection area.

  Note that the above-described threshold value calculation method is an example and is not limited to this, and can be arbitrarily changed as appropriate. For example, when the background color in the frame image is one color or the pattern is uniform, the threshold setting unit 5d is calculated for any one detection area (for example, the left detection area A1). The representative value (threshold value) may be applied to other detection areas (for example, the right detection area A2).

The frame number specifying unit 5e specifies the frame number when the subject is present in the detection area.
That is, the frame number specifying unit 5e determines the frame number (frame) at the time when the subject is present in each detection area based on the change in the pixel value of each detection area in the plurality of detection areas set by the area setting unit 5b. Frame number at the time of in or frame number at the time of frame out). Specifically, the frame number specifying unit (determination unit) 5e determines, for each detection region, whether the pixel value for each detection region is larger than each threshold set by the threshold setting unit 5d. When it is determined that the pixel value for each detection region is larger than each threshold set by the threshold setting unit 5d, the pixel value of the detection region at the time when the pixel value exceeds the threshold set by the threshold setting unit 5d. The frame number is specified (see FIGS. 6B and 6C).
For example, the frame number specifying unit 5e calculates, as the evaluation value, the difference square sum SSD of the pixel values (for example, luminance values) in the detection area of each frame image decoded by the decoding unit 5c with the adjacent frame image. Subsequently, the frame number specifying unit 5e compares the calculated evaluation value with the threshold set by the threshold setting unit 5d, and determines whether or not the evaluation value is larger than the threshold. As a result of the determination, the frame number specifying unit 5e specifies the frame number at the time when the evaluation value exceeds the threshold value. For example, in the case of a golf swing, the evaluation value becomes larger than the threshold value when the club head C moving in the left-right direction passes through the detection region.
Also, the frame number specifying unit 5e specifies the frame number at the time when the degree of difference with respect to the threshold is the largest among the frame numbers at the time when the evaluation value is determined to be larger than the threshold. That is, for example, when the club head C that moves in the left-right direction passes through the detection area, the evaluation value becomes larger than the threshold value, and when the overlapping area between the club head C and the detection area becomes the largest, the degree of difference is the largest. Become. Here, the evaluation value also becomes larger than the threshold value when the hit ball B passes through one detection region (a detection region A2 on the frame-out side described later), but the evaluation value when the club head C passes through. Since this is larger than the evaluation value of the ball B, it is considered that the club head C exists in the detection area of the frame image corresponding to the evaluation value having the largest difference with respect to the threshold value.

Here, the frame number identification processing at the time of frame-in and frame-out in the left and right detection areas A1 and A2 starts from the detection area A2 (see FIG. 6B) corresponding to the frame-out side of the subject. May be done. That is, for example, in the case of a golf swing, a right-handed person's swing draws a trajectory in which the club head C frames in from the left side and frames out from the right side. On the other hand, in the left-handed person's swing, on the contrary, the specific processing is performed first from the left detection area A1.
At this time, the frame number specifying unit 5e processes all frame images decoded by the decoding unit 5c for the frame number specifying process at the time of frame out using the detection area A2 (see FIG. 6B). set to target. On the other hand, the frame number specifying unit 5e uses the frame number at the time of the specified frame-out as a reference for the process of specifying the frame number at the time of frame-in using the detection area A1 (see FIG. 6B). A predetermined number of frame images are processed. For example, the frame number specifying unit 5e is specified using the detection area A2 in consideration of the standard swing speed (for example, 30 to 40 m / s) of golf, the imaging frame rate and the angle of view of the imaging unit 3, and the like. A frame image that is traced back a predetermined number of frames from the frame image corresponding to the frame number at the time of the specified frame-out is specified, and is a frame after the specified frame image and specified using the detection area A2. The frame image preceding the frame image corresponding to the frame number at the time of frame out is the processing target.
Thus, in the case of a golf swing, the take-back and swing-down operations pass through the detection area A1 on the frame-in side, but the club head C is within the detection area A1 during take-back. It is possible to prevent the frame number of an existing frame image from being erroneously specified as the frame number at the time of frame-in.

The image specifying unit 5f specifies a plurality of frame images based on a predetermined time point in the movement of the subject.
That is, the image specifying unit (specifying unit) 5f determines a predetermined time point (for example, in the motion of the subject in the plurality of images) based on the result of specifying the frame number at the time of frame-in and frame-out by the frame number specifying unit 5e. A plurality of frame images related to the impact at the moment of hitting the ball B by the club head C) are specified. Specifically, for example, the image specifying unit 5f, based on the result of specifying the frame number at the time of frame-in and frame-out of the subject (for example, the club head C) using the detection areas A1 and A2, And frame images F2 and F4 corresponding to the frame numbers at the time of frame-out are specified. Then, the image specifying unit 5f specifies a plurality of frame images based on the predetermined time point among the plurality of frame images based on the frame images F2 and F4 related to the predetermined time point. For example, the image specifying unit (extraction unit) 5f specifies a plurality of frame images between the two frame numbers from the plurality of frame images based on the two specified frame numbers, and extracts them as moving images.
At this time, the image specifying unit 5f, when the order of the frame numbers at the time of frame-in and frame-out specified using the plurality of detection areas by the frame number specifying unit 5e is a predetermined order, The frame image related to the is specified. In other words, when the frame number at the time of frame-in specified using the detection area A1 is smaller than the frame number at the time of frame-out specified using the detection area A2, the two frame numbers are set. Between the corresponding frame images F2 and F4, a frame image F3 corresponding to the impact time (predetermined time) at the moment of hitting the ball B by the club head C is included.

  The image specifying unit 5f considers the relative positional relationship between the position of the ball B or the club head in the initial state and the two detection areas A1 and A2, for example, between the two frame images F2 and F4. The frame image F3 itself corresponding to the impact at the moment of hitting the ball B may be specified. For example, when the ball B is arranged approximately in the middle of the two detection areas A1 and A2, the frame image F3 corresponding to the impact at the moment of hitting the ball B approximately in the middle between the two frame images F2 and F4. Is considered to exist.

The image recording unit 6 is configured by, for example, a nonvolatile memory (flash memory). The image recording unit 6 records image data of various images encoded by a coding unit (not shown) of the image data generation unit 4 using a predetermined coding method.
Specifically, the image recording unit 6 is, for example, image data of a moving image in which a series of movements of a subject are continued by the imaging unit 3, or a predetermined number of images extracted from the moving image and based on a predetermined time point. Image data of a moving image composed of frame images is recorded.

  The image recording unit 6 may be configured, for example, such that a recording medium (not shown) is detachable and controls reading of data from the mounted recording medium and writing of data to the recording medium.

  The display unit 7 displays still images and moving images. Specifically, the display unit 7 includes a display panel 7a and a display control unit 7b.

The display panel 7a displays an image in the display area. Specifically, the display unit 7 updates the live view image while sequentially updating a plurality of image frames generated by imaging the subject by the imaging unit 3 in the still image capturing mode and the moving image capturing mode. L is displayed.
Examples of the display panel 7a include a liquid crystal display panel and an organic EL display panel, but are not limited to these examples.

  The display control unit 7b performs control to display a predetermined image on the display screen of the display panel 7a based on the image data of a predetermined size read from the image recording unit 6 and decoded by the image processing unit 5. Specifically, the display control unit 7b includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. The digital video encoder reads out the luminance signal Y and the color difference signals Cb and Cr decoded by the image processing unit 5 and stored in the VRAM (not shown) from the VRAM via the VRAM controller, and stores these data. Originally, a video signal is generated and output to the display panel 7a.

The operation input unit 8 is for inputting various instructions to the apparatus main body.
Specifically, the operation input unit 8 includes, for example, an operation unit (not shown) including up / down / left / right cursor buttons and a determination button related to a selection instruction of a mode, a function, and the like.
When the user operates various buttons on the operation unit, the operation input unit 8 outputs an operation instruction corresponding to the operated button to the central control unit 1. The central control unit 1 causes each unit to execute a predetermined operation (for example, imaging of a subject) in accordance with an operation instruction output from the operation input unit 8 and input.
The operation input unit 8 may include a touch panel (not shown) provided integrally with the display panel 7a of the display unit 7.

Next, imaging processing will be described with reference to FIGS.
FIG. 2 is a flowchart illustrating an example of an operation related to the imaging process.

  The imaging processing described below is executed when an impact image extraction mode is selected from a plurality of operation modes displayed on a menu screen (not shown) based on a predetermined operation of the operation input unit 8 by the user. Process.

<Imaging process>
As shown in the flowchart of FIG. 2, the display control unit 7b first displays an index M (for example, a club head shape or a ball shape frame display) for aligning the initial positions of the club head C and the ball B, for example. Overlaid on the view image L, OSD display is performed on the display panel 7a (step S1; see FIG. 4). In this state, the imager adjusts the distance between the apparatus main body and the subject, the angle of view of the imaging unit 3, and the like so that the initial positions of the club head C and the ball B overlap the index M.
Then, when an instruction to complete adjustment of the distance between the apparatus main body and the subject, the angle of view of the imaging unit 3, and the like is input based on a predetermined operation of the operation input unit 8 by the user, the display control unit 7b displays the index M. The OSD display of is stopped.
The OSD display of the index M is prepared according to the dominant hand of the person who operates, and FIG. 4 is for right-handed persons. Further, the subject may be imaged with the apparatus main body fixed to a tripod, or may be performed by the imager by hand.

Next, when imaging is instructed based on a predetermined operation of the operation input unit 8 by the user, the imaging unit 3 captures a golf swing motion at a predetermined imaging frame rate (for example, 1000 fps), The data generation unit 4 generates image data of a moving image obtained by encoding a plurality of frame images using a predetermined encoding method (step S2).
Then, the image data generation unit 4 transfers the generated moving image image data to the memory 2.

Subsequently, the image processing unit 5 performs an image specifying process (see FIG. 3) for specifying and extracting a predetermined number of frame images from the image data of the moving image (step S3).
The image specifying process will be described later.

Then, the display control unit 7b displays a moving image on the display panel 7a based on the predetermined number of frame images specified and extracted by the image specifying unit 5f (step S4).
Thereby, the imaging process is terminated.

<Image identification processing>
Hereinafter, the image specifying process will be described in detail with reference to FIG.
FIG. 3 is a flowchart illustrating an example of an operation related to the image specifying process.

As shown in FIG. 3, the image acquisition unit 5 a reads out image data of a moving image composed of a plurality of frame images obtained by continuously capturing golf swing motions from the memory 2, and uses it as a processing target for image specifying processing. Obtain (step S11).
Next, the region setting unit 5b sets two substantially rectangular detection regions at positions near both ends in the left-right direction at the substantially central portion in the vertical direction for each of the plurality of frame images constituting the image data of the moving image ( Step S12). Here, in the case of a right-handed person, the left detection area A1 is the frame-in side, and the right detection area A2 is the frame-out side.

  Subsequently, the decoding unit 5c decodes a predetermined number (for example, n; n is a natural number) of frame images from the first frame image F1 constituting the moving image data (step S13), and a threshold setting unit 5d calculates a difference square sum SSD of pixel values (for example, luminance values) with the adjacent frame images for each of the left and right detection areas A1 and A2 (step S14). Then, the threshold value setting unit 5d calculates a representative value (for example, an average value or a median value) of a plurality of (for example, n-1) difference square sums SSD for each of the left and right detection areas A1 and A2. It is set as a threshold value for specifying the frame number at the time of frame-in and frame-out (step S15).

Next, the decoding unit 5c sequentially decodes all the frame images constituting the image data of the moving image from the first frame image F1 (step S16), and the frame number specifying unit 5e determines the decoded frame image. Then, the difference square sum SSD of the pixel values (for example, luminance values) in the detection area A2 corresponding to the frame-out side of the previous frame image is sequentially calculated as an evaluation value (step S17). Subsequently, the frame number specifying unit 5e determines whether or not the calculated evaluation value is larger than the threshold value of the detection area A2 corresponding to the frame-out side set by the threshold value setting unit 5d (step S18).
Here, if it is determined that the evaluation value is not greater than the threshold value (step S18; NO), the image processing unit 5 returns the process to step S16, and executes each subsequent process. That is, it is considered that the club head C does not pass through the detection area A2 because there is no change in the pixel value in the detection area A2 corresponding to the frame-out side. On the other hand, if the pixel value varies within the detection area A2 corresponding to the frame-out side, it is considered that the club head C exists in the detection area A2.

If it is determined in step S18 that the evaluation value is larger than the threshold value (step S18; YES), the frame number identification unit 5e calculates the degree of difference between the evaluation value and the threshold value, and the degree of difference is the largest. The frame number at the time corresponding to the evaluation value is specified, and the frame number for identifying the frame image corresponding to the frame number is updated with the frame number of the frame image, and predetermined storage means (for example, the memory 2 or the like) ) Temporarily (step S19).
Subsequently, the image processing unit 5 determines whether or not each of the above processes has been performed for all the frame images (step S20).

If it is determined in step S20 that all frame images have not been performed (step S20; NO), the image processing unit 5 returns the process to step S16, and executes the subsequent processes.
Each of the above processes is repeatedly executed until it is determined in step S20 that the process has been performed for all frame images (step S20; YES).
As a result, the frame image of the frame number at the time when the difference between the evaluation value and the threshold value is maximized due to the largest overlap area between the club head C moving in the left-right direction and the detection area A2 on the frame-out side. The frame number of F4 (see FIG. 6B) is specified.

Next, the decoding unit 5c designates a frame image that goes back a predetermined number of frames from the frame image F4 corresponding to the frame number at the time of frame-out specified using the detection area A2, as a decoding start frame image. (Step S21). In addition, the decoding unit 5c performs decoding processing up to a frame image after the designated start frame image and up to the frame image immediately preceding the frame image F4 corresponding to the frame number at the time of the specified frame-out. Specify as an image.
Then, the decoding unit 5c sequentially decodes the frame images after the designated start frame image (step S22), and the frame number specifying unit 5e sets the decoded frame image to the previous frame image. The difference square sum SSD of the pixel values (for example, luminance values) of the detection area A1 corresponding to the frame-in side is sequentially calculated as an evaluation value (step S23). Subsequently, the frame number specifying unit 5e determines whether or not the calculated evaluation value is larger than the threshold value of the detection area A1 corresponding to the frame-in side set by the threshold value setting unit 5d (step S24).
Here, if it is determined that the evaluation value is not greater than the threshold value (step S24; NO), the image processing unit 5 returns the process to step S22, and executes the subsequent processes. That is, it is considered that the club head C does not pass through the detection area A2 because there is no change in the pixel value in the detection area A1. On the other hand, it is considered that the club head C exists when the pixel value varies within the detection area A1.

If it is determined in step S24 that the evaluation value is larger than the threshold value (step S24; YES), the frame number identification unit 5e calculates the difference between the evaluation value and the threshold value, and the difference is the largest. The frame number corresponding to the evaluation value is specified, the frame number for identifying the frame image on the frame-in side is updated with the frame number of the frame image, and temporarily stored in a predetermined storage means (for example, the memory 2). (Step S25).
Subsequently, the image processing unit 5 determines whether or not each of the above processes has been performed for all the frame images to be processed (step S26).

If it is determined in step S26 that all the frame images to be processed have not been performed (step S26; NO), the image processing unit 5 returns the process to step S22, and executes the subsequent processes.
Each of the above processes is repeatedly executed until it is determined in step S26 that the process has been performed for all the frame images to be processed (step S26; YES).
As a result, the frame image F2 (see FIG. 6C) in which the degree of difference between the evaluation value and the threshold value is maximized due to the largest overlap area between the club head C moving in the left-right direction and the detection area A1. A frame number is specified.

Next, the image specifying unit 5f specifies a predetermined number of frame images between the two frame numbers from a plurality of frame images based on the specified frame numbers on the frame-in side and the frame-out side, Extracted as an image (step S27). Specifically, for example, the image specifying unit 5f starts from a frame image having a frame number that is a predetermined number before the frame number on the specified frame-in side, a predetermined number after the frame number on the specified frame-out side. Up to the frame number frame image is extracted as a moving image.
Here, the interval between the frame number on the frame-in side and the frame number on the frame-out side, that is, the number of images between the two frame images F2 and F4 is larger than the predetermined value, or the corresponding time interval is larger than the predetermined value. If it is long, there is a possibility that something other than the club head C has passed through the detection region, so the image specifying unit 5f extracts a moving image when the number of frame images and the time interval are within predetermined values. It may be a configuration.

  Thereafter, the image recording unit 6 records the image data of the moving image composed of a predetermined number of frame images with reference to a predetermined time point extracted by the image specifying unit 5f (step S28), and performs the image specifying process. finish.

As described above, according to the imaging apparatus 100 of the present embodiment, detection areas are set at a plurality of predetermined positions in each frame image of a plurality of frame images in which the motion of the subject is continuously captured, and the set plurality Since a plurality of frame images related to a predetermined point in time of the operation of the subject are identified among the plurality of frame images based on fluctuations in pixel values between the plurality of frame images in each of the detection regions, each of the plurality of frame images By using the change in the pixel value of the detection area set at the predetermined position, it is possible to specify user-desired frame images (for example, frame images F2 to F4) without using a special device.
For example, in the case of a golf swing, it is possible to specify a user-desired frame image including a frame image that will be the moment of impact if the ball B is hit, even if the ball B is not actually hit. it can.

Further, when a frame image in which a subject is present in each of the plurality of detection areas is specified, and the frame number of the frame image in which the subject is present in each specified detection area is in a predetermined order, Since a plurality of related frame images are specified, it is possible to determine whether or not the motion of the subject is other than a predetermined motion using the order of detection of the subject from the plurality of detection areas.
For example, in the case of a golf swing action, generally, the detection area A1 on the frame-in side (for example, the left side in a right-handed swing) moves to the detection area A2 on the frame-out side (for example, the right side in a right-handed swing). And the club head C operates. On the other hand, when the club head C is simply moved to the left and right, the club head C is detected in the detection area A1 on the frame-in side after the club head C is detected in the detection area A2 on the frame-out side. Although there is a possibility that the club head C is detected using a plurality of detection areas, the detection area A1 on the frame-in side is first, and the detection area A2 on the frame-out side is later. By specifying a plurality of frame images related to, it is possible to suppress erroneously specifying a frame image corresponding to an operation of simply moving the club head C to the left or right (for example, waggle or the like).

Further, among the plurality of frame images, a representative value of the pixel values of the detection region in a predetermined number of images within a predetermined period is set as a threshold for specifying the frame number at the time of frame-in or frame-out, and the threshold is set. A frame number at the time of frame-in or frame-out can be specified as a reference, and a frame image in which a subject exists in each detection region can be specified. In particular, when it is determined that the pixel value of each detection area is larger than the threshold value, the frame number at the time of frame-in or frame-out can be properly detected by the detection area. An existing frame image can be specified as a frame image related to a predetermined time point.
That is, for example, when the subject does not exist in the detection region, the pixel value of the detection region hardly changes, whereas when the subject exists in the detection region, the pixel value of the detection region changes. Therefore, the frame number at the time of frame-in or frame-out can be properly detected by using the representative value of the pixel value of the detection region as a threshold and using the variation of the pixel value of the detection region.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in the above embodiment, the frame number at the time of frame-in and frame-out is specified based on the threshold value set in the detection area. However, this is an example, and the present invention is not limited to this. Whether or not to set a threshold value when specifying the frame number at the time of frame-out can be arbitrarily changed as appropriate. That is, the imaging apparatus 100 does not necessarily have to include the threshold setting unit 5d as long as the frame number at the time of frame-in and frame-out can be specified based on the change in the pixel value of the detection area.

  In the above embodiment, the frame number identification process at the time of frame-in and frame-out in the two left and right detection areas A1 and A2 is performed first from the detection area A2 corresponding to the frame-out side of the subject. However, this is only an example, and the present invention is not limited to this. For example, the detection may be performed first from the detection area A1 corresponding to the frame-in side of the subject. In this case, the frame number specifying unit 5e sets all frame images decoded by the decoding unit 5c as processing targets for the frame number specifying process at the time of frame-in by the detection region A1, while using the detection region A2. The frame number identifying process at the time of frame-out may be based on the frame number at the time of frame-in.

Further, the frame number specifying unit 5e specifies the frame number having the largest difference from the threshold among the frame numbers determined to have the evaluation value larger than the threshold. When there are a plurality of evaluation values close to the largest evaluation value, the frame number at the time of frame-in and frame-out may be specified by performing a predetermined process.
That is, in specifying the frame number at the time of frame-in, a process of specifying the last frame number as the frame number at the time of frame-in among the frame numbers of a plurality of evaluation values adjacent to the largest evaluation value. For example, it is possible to suppress erroneous detection of an address-top operation as a frame number at a desired frame-in among address-top and top-impact operations in a swing operation. it can. Similarly, in specifying the frame number at the time of frame-out, the last frame number is specified as the frame number at the time of frame-out among the frame numbers of a plurality of evaluation values adjacent to the largest evaluation value. By adding such processing, it is possible to suppress erroneous detection of the frame number at the time when the ball is detected using the detection area after the impact as the frame number at the desired frame-out time.

  Furthermore, in the above-described embodiment, the image specifying unit 5f has the identification information (for example, frame-in and The frame images F2 and F4 corresponding to the identification information are specified after specifying the frame number at the time of frame-out. However, this is only an example, and the frame images F2 and F4 are not necessarily limited thereto. There is no need to specify. In addition, the image specifying unit 5f directly selects a frame image related to a predetermined point in the motion of the subject from a plurality of frame images constituting the image data of the moving image directly based on the specifying result by the frame number specifying unit 5e. You may specify.

  Furthermore, the configuration of the imaging apparatus 100 is merely an example illustrated in the above embodiment, and is not limited thereto. Furthermore, although the imaging device 100 is illustrated as an image processing device, it is not limited to this.

In addition, in the above-described embodiment, the functions as the acquisition unit, the region setting unit, and the specifying unit are controlled by the image acquisition unit 5a, the region setting unit 5b, and the image specifying unit 5f under the control of the central control unit 1. The configuration realized by driving is not limited to this, and may be realized by executing a predetermined program or the like by the CPU of the central control unit 1.
That is, a program memory (not shown) that stores a program stores a program including an acquisition processing routine, an area setting processing routine, and a specific processing routine. Then, the CPU of the central control unit 1 may function as a means for acquiring a plurality of images obtained by continuously capturing the motion of the subject by an acquisition process routine. Further, the CPU of the central control unit 1 may function as means for setting an area at a plurality of predetermined positions in each of a plurality of acquired images by an area setting processing routine. In addition, the CPU of the central control unit 1 at a predetermined time in the operation of the subject among the plurality of images based on the variation of the pixel value between the plurality of images in each of the plurality of set areas by the specific processing routine. You may make it function as a means to specify two or more related images.

  Similarly, the threshold setting unit, the determination unit, the calculation unit, and the extraction unit may be realized by executing a predetermined program or the like by the CPU of the central control unit 1.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Acquisition means for acquiring a plurality of images in which the motion of the subject is continuously captured;
Area setting means for setting areas at a plurality of predetermined positions in each of the plurality of images acquired by the acquisition means;
Based on fluctuations in pixel values between the plurality of images in each of the plurality of regions set by the region setting means, a plurality of images related to a predetermined time point in the operation of the subject are specified from the plurality of images. Specific means,
An image processing apparatus comprising:
<Claim 2>
The specifying means further includes:
Based on fluctuations in pixel values between the plurality of images in each of the plurality of regions, an image in which a subject exists in each of the plurality of regions is specified, and an image in which the subject exists in each specified region The image processing apparatus according to claim 1, wherein a plurality of images related to the predetermined time point are specified when the frame numbers are in a predetermined order.
<Claim 3>
A threshold setting unit configured to set a representative value of pixel values of the region in a predetermined number of images within a predetermined period of the plurality of images as a threshold for detecting a subject;
The image processing apparatus according to claim 2, wherein the specifying unit specifies an image in which a subject is present in each of the regions based on the threshold set by the threshold setting unit.
<Claim 4>
Determination means for determining whether or not the pixel value of each region is larger than a threshold set by the threshold setting means;
When the determination unit determines that the pixel value of each region is larger than the threshold value, the specifying unit specifies an image in which a subject is present in the region as an image related to the predetermined time point. The image processing apparatus according to claim 3.
<Claim 5>
A calculation unit that calculates an average value or a median value of the pixel values of the region in a predetermined number of images from the plurality of images acquired by the acquisition unit;
The image processing apparatus according to claim 3, wherein the threshold value setting unit sets an average value or a median value calculated by the calculation unit as the representative value.
<Claim 6>
6. The image processing apparatus according to claim 1, further comprising an extraction unit that extracts a plurality of images related to a predetermined time point specified by the specifying unit as a moving image.
<Claim 7>
An image processing method using an image processing apparatus,
Processing to acquire a plurality of images in which the motion of the subject is continuously captured;
A process of setting a region at a plurality of predetermined positions in each of a plurality of acquired images;
A process of specifying a plurality of images related to a predetermined time point in the operation of the subject among the plurality of images based on fluctuations in pixel values between the plurality of images in each of the plurality of regions set;
An image processing method comprising:
<Claim 8>
The computer of the image processing device
Acquisition means for acquiring a plurality of images obtained by continuously capturing the motion of the subject;
Area setting means for setting areas at a plurality of predetermined positions in each of the plurality of images acquired by the acquisition means;
Based on fluctuations in pixel values between the plurality of images in each of the plurality of regions set by the region setting means, a plurality of images related to a predetermined time point in the operation of the subject are specified from the plurality of images. Specific means,
A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Central control part 5a Image acquisition part 5b Area setting part 5d Threshold setting part 5e Frame number specific | specification part 5f Image specific part A1, A2 Detection area

Claims (13)

Acquisition means for acquiring a plurality of images in which the motion of the subject is continuously captured;
A region setting unit that sets a plurality of partial regions that are smaller than the entire region of the image and are not superimposed on each of the plurality of images acquired by the acquisition unit;
Based on fluctuations in pixel values between the plurality of images in each of the plurality of partial regions set by the region setting means, a plurality of images related to a predetermined time point in the movement of the subject are identified among the plurality of images. Specific means to
An image processing apparatus comprising: The specifying means further includes:
Based on the pixel value variation between the plurality of images in each of the plurality of partial areas, an image in which the subject is present in each of the plurality of partial areas is identified, and the subject is present in each identified partial area. The image processing apparatus according to claim 1, wherein a plurality of images related to the predetermined time point are specified when the frame numbers of the images to be processed are in a predetermined order. A threshold setting unit configured to set a representative value of pixel values of the partial area in a predetermined number of images within a predetermined period of the plurality of images as a threshold for detecting a subject;
The image processing apparatus according to claim 2, wherein the specifying unit specifies an image in which a subject is present in each of the partial areas based on the threshold set by the threshold setting unit. Determination means for determining whether or not the pixel value of each partial region is larger than a threshold set by the threshold setting means;
When the determination unit determines that the pixel value of each partial region is larger than the threshold value, the specifying unit specifies an image in which a subject exists in the partial region as an image related to the predetermined time point. The image processing apparatus according to claim 3. A calculation unit that calculates an average value or a median value of the pixel values of the partial areas in a predetermined number of images from the plurality of images acquired by the acquisition unit;
The image processing apparatus according to claim 3, wherein the threshold value setting unit sets an average value or a median value calculated by the calculation unit as the representative value.   6. The image processing apparatus according to claim 1, further comprising an extraction unit that extracts a plurality of images related to a predetermined time point specified by the specifying unit as a moving image. The specifying unit specifies an image associated with a start time and an end time of the moving image extraction based on the motion of the subject among the plurality of images,
The image processing apparatus according to claim 6, wherein the extracting unit extracts a plurality of images in a range corresponding to each image specified by the specifying unit as the moving image.   The image processing apparatus according to claim 6, wherein the extracting unit extracts a moving image when a plurality of images related to the predetermined time point specified by the specifying unit are within a predetermined number. .   The image processing apparatus according to claim 7, wherein the extracting unit extracts a moving image when a range corresponding to the image specified by the specifying unit is within a predetermined length.   The image according to any one of claims 1 to 9, wherein the region setting means sets the partial region at a plurality of predetermined positions in each of a plurality of images based on the motion of the subject. Processing equipment.   The image processing apparatus according to claim 10, wherein the area setting unit sets the partial area at a plurality of predetermined positions in each of a plurality of images based on a direction of movement of the subject. An image processing method using an image processing apparatus,
Processing to acquire a plurality of images in which the motion of the subject is continuously captured;
To each of the obtained plurality of images, the processing range than the entire area of the image is small, that and setting a plurality of partial areas which does not overlap,
A process of specifying a plurality of images related to a predetermined time point in the operation of the subject among the plurality of images based on fluctuations in pixel values between the plurality of images in each of the set partial areas;
An image processing method comprising: The computer of the image processing device
Acquisition means for acquiring a plurality of images obtained by continuously capturing the motion of the subject;
Wherein each acquisition unit plurality of images acquired by a smaller range than the entire area of the image, and area setting means for setting a plurality of partial areas which does not overlap,
Based on fluctuations in pixel values between the plurality of images in each of the plurality of partial regions set by the region setting means, a plurality of images related to a predetermined time point in the movement of the subject are identified among the plurality of images. Specific means to
A program characterized by functioning as

Images