JP2004180289A - Image pickup apparatus and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily control reproduction of content by easily deciding which frame is used to pick up an image. <P>SOLUTION: An image pickup unit reads a signal from an image pickup element to generate an image pickup signal based on a picked-up image. A signal processing unit generates video data from the image pickup signal. An analog audio signal is sampled to generate audio data. A control unit controls the operations of the image pickup unit and signal processing unit to vary a frame rate of the video data into a desired preset frame rate and generates associated information indicating this preset frame rate etc. An output unit combines the associated information with the video data and the audio data, and outputs the combined data as material data. <P>COPYRIGHT: (C)2004,JPO

Description

  The present invention relates to an imaging device and an imaging method. Specifically, video data is generated based on a signal read from the image sensor, the frame rate of the video data is variably set to a desired setting frame rate, and additional information including frame rate information indicating the setting frame rate is generated. In this case, additional information is linked to video data and output.

  2. Description of the Related Art Conventionally, in the generation of image and / or audio content used for broadcasting or the like, in order to produce an effect intended by a creator, it is often the case that content in which the speed of movement of a subject is partially changed for a certain period is created. .

  In the generation of the content in which the movement speed is changed, for example, the content is generated by setting the frame rate higher than the reference frame rate, and the content is reproduced at the reference frame rate, so that the motion is expressed slower than actual. Generated content. In addition, content is generated by setting the frame rate to be lower than the reference frame rate, and the content is reproduced at the reference frame rate, thereby generating content expressing motion faster than it actually is. Further, by adjusting the set frame rate or the frame rate at the time of reproduction, the speed of the movement can be freely varied.

  In this way, the creator can use the content generated at the reference frame rate as well as the content whose speed of movement is changed so that the intended effect can be obtained when played back at the reference frame rate. Generate

  A video camera capable of expanding and compressing the time axis so as to generate content with a variable frame rate as described above is disclosed in, for example, Patent Document 1.

JP-A-11-177903

  By the way, when the content generated at the reference frame rate and the content whose movement speed is changed as described above are used as the material to generate the content to be used for broadcasting or the like, simply displaying the image of the material is not enough for each content. At what frame rate it is generated cannot be easily identified at the time of editing. Further, in the case of performing the speed change process on the content whose motion speed has been changed, it is not possible to determine whether the speed change can be easily performed without deteriorating the image quality. For example, if the set frame rate (hereinafter, referred to as “set frame rate”) is set to 10 times the reference frame rate, when the content generated at the set frame rate is reproduced at the reference frame rate, the motion is reduced by 1 / The content is expressed as 10 times speed. Here, when it is desired to express the motion as 1/5 times speed, since the set frame rate is 10 times higher than the reference frame rate, by thinning out every frame, the speed can be easily reduced without deteriorating the image quality. Identify that changes can be made. However, it is not possible to identify whether or not the speed can be easily changed without deteriorating the image quality only by the displayed image.

  In view of the above, the present invention provides an image pickup apparatus and an image pickup method capable of easily identifying at what frame rate an image was picked up.

  An imaging apparatus according to the present invention reads out a signal from an imaging element and generates an imaging signal based on a captured image, a signal processing unit that generates video data based on the imaging signal, an imaging unit, and a signal processing unit. Control means for controlling the operation of at least one of the means to variably set the frame rate of the video data to a desired set frame rate and generating at least additional information indicating the set frame rate; And output means for connecting and outputting.

  Also, the imaging method according to the present invention includes a step of generating video data based on a signal read from the image sensor, a step of variably setting a frame rate of the video data to a desired set frame rate, and It has a step of generating additional information including frame rate information, and a step of connecting the additional information to video data and outputting it.

  In the present invention, by reading a signal from the image sensor, generating an image signal based on the captured image, and when generating video data based on the image signal, by changing the signal read timing from the image sensor, By changing the frame rate of the video data, or by thinning out the frames, the frame rate of the video data is changed, or the signal readout timing from the image sensor is changed, and the frame addition is performed, so that the video data Set the frame rate variably. The frame rate of the video data is variably set to a desired set frame rate, and frame rate information indicating the set frame rate is generated. The video data indicating the image is set as main data, and a frame The attached information including the rate information is connected and output. Further, the attached information includes a subframe number added to a frame of a set frame rate included in a frame period of a predetermined reference frame rate. Further, audio data is generated by sampling the analog audio signal, and the sampling cycle is controlled based on the set frame rate. The video data and the audio data representing the image and the audio are set as main data, and the main data is connected to the attached information and output. In the output of the main data, the main data in which the additional information is linked to a recording medium such as a video tape or a disk-shaped recording medium is recorded.

  According to the present invention, video data is generated based on a signal read from the image sensor, and the frame rate of the video data is variably set to a desired set frame rate. Auxiliary information including frame rate information indicating the set frame rate is generated, and is output in connection with video data. For this reason, variable speed reproduction can be easily performed using this frame rate information.

  Also, since the frame rate of the video data is variably set by variably setting the signal read timing from the image sensor, the frame rate of the video data can be freely changed. Further, since the frame rate of the video data is variably set by performing frame thinning, the configuration of the imaging apparatus can be simplified. Furthermore, by variably setting the signal readout timing from the image sensor and performing frame addition, the frame rate of the video data is variably set, so that the frame rate of the video data can be freely changed and the signal readout timing from the imager can be changed. Can be reduced.

  In addition, a subframe number is added to a frame having a set frame rate included in a frame period of a predetermined reference frame rate, and the subframe number is included in the attached information. When the speed is variably set, the thinning of the video data can be easily performed.

  Further, the analog audio signal is sampled to generate audio data, the sampling period of the analog audio signal is controlled based on the set frame rate, and the video data and the audio data are connected with the attached information and output. Therefore, not only the image but also the sound can be reproduced at a variable speed according to the image.

  In addition, since the auxiliary data including the frame rate information of the main data is connected to the main data indicating the image and / or the sound and recorded and output on the recording medium, the recording medium is reproduced and the frame rate information is output. The variable speed reproduction that is used can be easily performed.

  Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows the overall configuration of a content providing system that distributes content, for example, image and / or audio content. The imaging device 10 generates video data with a variably set frame rate, and connects the accessory information including the frame rate information indicating the frame rate of the video data to the video data to generate the material data DTm. Alternatively, it is supplied to the editing device 30. Further, when the audio input device 19 is provided in the imaging device 10, the audio data is generated and supplied to the signal recording device 20 or the editing device 30 as the material data DTm together with the accessory information.

  The signal recording device 20 records the supplied material data DTm on a recording medium. Further, the material data DTm recorded on the recording medium is reproduced and supplied to the editing device 30. Note that the imaging device 10 and the signal recording device 20 may be of a so-called dockable type and can be integrated in addition to a completely integrated type, and the imaging device 10 and the signal recording device 20 may be used separately. And may be connected via a cable.

  The editing device 30 performs an editing process using the material data DTm supplied from the imaging device 10 or the signal recording device 20, and generates video data and audio data desired by the editor. Further, the editing device 30 generates additional content data DC for broadcast or distribution, by linking the generated video data and audio data with additional information including frame rate information and the like.

  FIG. 2 shows the configuration of the imaging device 10. The light incident through the imaging lens system 11 is incident on the imaging unit 12, and a subject image is formed on an imaging surface of an imaging element such as a CCD (Charge Coupled Device) provided in the imaging unit 12. . The imaging element generates an imaging charge of a subject image by photoelectric conversion. Further, based on a driving signal CR from a timing generator 142 described later, the generated imaging charge is read out, an imaging signal Sp having a frame rate corresponding to the driving signal CR is generated, and a camera processing circuit 131 of the signal processing unit 13 is generated. To supply.

  The camera processing circuit 131 performs various signal processing at a timing synchronized with the imaging signal Sp based on the timing signal CT supplied from the timing generator 142. For example, a process of removing noise components from the imaging signal Sp by performing correlated double sampling, a process of converting the imaging signal Sp from which noise has been removed into digital video data, a clamping process of video data, a shading correction, an imaging device Correction processing, gamma processing, contour compensation processing, knee correction processing, and the like. In addition, various signal processes are performed under processing conditions based on the operation control signal CS supplied from the imaging control circuit 141 of the control unit 14. As described above, the video data DV obtained by performing various kinds of signal processing by the camera processing circuit 131 is supplied to the output unit 15.

  The timing generator 142 of the control unit 14 generates a drive signal CR according to the operation control signal CS from the imaging control circuit 141 and supplies the driving signal CR to the imaging unit 12, thereby changing the readout cycle of the imaging charge in the imaging unit 12. Then, the frame rate of the imaging signal Sp is controlled to the set frame rate FRs based on the operation signal PSa from the user interface unit 16. For example, in the NTSC system, the frame frequency is 59.94 Hz or 29.97 Hz, in the PAL system, the frame frequency is 50 Hz or 25 Hz as the frame frequency of the reference frame rate FRr, and the set frame rate FRs is k (k is not limited to an integer) of the reference frame rate FRr. When the operation of multiplying by (positive value) is performed, control is performed so that the frame rate of the image signal Sp becomes k times the reference frame rate FRr. Note that, for example, by changing the period of a read pulse (sensor gate pulse) for moving the imaging charge accumulated in each pixel from each pixel of the imaging device such as a CCD to the transfer unit, the imaging charge is read. The cycle is varied, and the frame rate is varied. At this time, a CDR (Common Data Rate) method may be adopted. When the CDR method is used, the effective frame rate is variable, but the frame rate of the signal output from the CCD can be unchanged, and the processing rate of the camera processing circuit 131 and the like can be constant. This CDR system is disclosed in PCT application, application number PCT / JP03 / 0051, and application on January 22, 2003.

  Further, the timing generator 142 generates a timing signal CT synchronized with the drive signal CR and supplies the timing signal CT to the camera processing circuit 131 and the audio processing circuit 132. Further, the timing generator 142 generates frame rate information DM-FRs indicating a set frame rate FRs which is a frame rate of the video data DV, and supplies the frame rate information DM-FRs to the output unit 15. The timing generator 142 also generates a subframe number BN. The sub-frame number BN is a number that makes it possible to identify each frame included in each frame period of the reference frame rate FRr when the set frame rate FRs is set higher than the reference frame rate FRr. This subframe number BN is supplied to the output unit 15 as frame identification information DM-BN.

  FIG. 3 is a flowchart showing an operation of adding a subframe number in the timing generator 142. The timing generator 142, for example, divides an oscillation signal of a predetermined frequency, sets the frame period of the reference frame rate FRr and the frame period of the set frame rate FRs in synchronization, and drives based on the frame period of the set frame rate FRs. The signal CR and the frame reference timing indicating the break of the frame period of the reference frame rate FRr are generated.

  In step ST1, it is determined whether or not the frame reference timing has been detected. Here, when the reference timing is detected, the process proceeds to step ST2. If the reference timing has not been detected, the process returns to step ST1.

  When the frame reference timing is detected in step ST1 and the process proceeds to step ST2, the subframe number BN is initialized in step ST2, the subframe number BN is set to an initial value, for example, “0”, and the process proceeds to step ST3.

  In step ST3, it is determined whether or not the frame reference timing has been detected before the elapse of one frame period of the set frame rate FRs from the detection of the frame reference timing. If the frame reference timing has not been detected, the process proceeds to step ST4, where "1" is added to the subframe number BN, the subframe number BN is updated, and the process returns to step ST3. As described above, when the frame reference timing is not detected before the elapse of one frame period of the set frame rate FRs, the subframe number BN is sequentially assigned for each one frame period of the set frame rate FRs.

  Thereafter, when the frame reference timing is detected before the elapse of one frame period of the set frame rate FRs, the process returns to step ST2, and the subframe number BN is initialized.

  Therefore, for each frame period of the reference frame rate FRr, the sub-frame number BN can be added to the frame image of the set frame rate FRs provided during this frame period.

  The user interface unit 16 is connected to the imaging control circuit 141 of the control unit 14 shown in FIG. When an operation switching operation or a variable frame rate operation is performed on the imaging device 10, the user interface unit 16 generates an operation signal PSa corresponding to the operation and supplies the operation signal PSa to the imaging control circuit 141. When the operation signal PSa is supplied from an external device (not shown) such as a remote controller, the user interface unit 16 supplies the operation signal PSa to the imaging control circuit 141.

  The imaging control circuit 141 generates an operation control signal CS based on the operation signal PSa from the user interface unit 16 so that the operation of the imaging device 10 becomes an operation corresponding to the operation signal PSa, Supply to generator 142.

  The audio processing circuit 132 is supplied with an analog audio signal Sin from the audio input device 19. The audio processing circuit 132 performs sampling processing of the audio signal Sin based on the timing signal CT supplied from the timing generator 142, generates digital audio data DA, and supplies the digital audio data DA to the output unit 15.

  The output unit 15 generates the additional information DM including the frame rate information DM-FRs and the frame identification information DM-BN, and connects the generated information to the video data DV and the audio data DA to generate the material data DTm. Alternatively, it is supplied to the editing device 30. Further, the additional information DM may include not only the information of the set frame rate FRs and the sub-frame number BN but also information indicating an imaging date and time, an imaging condition, an imaging content, and the like.

  Here, as an example of a method of connecting the additional information DM to the video data DV and the audio data DA, when the video data DV and the audio data DA are compressed to generate the material data DTm as a data stream, the data The additional information DM may be inserted into the header of the data stream, or the additional information DM may be inserted into the header of the data stream. For example, when the moving picture compression method of MPEG (Moving Picture Experts Group) standardized as ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) 13818-2 is used, the extension provided in the picture layer is used. By inserting the additional information DM in the area or the like, the additional information DM can be inserted in units of pictures. When the synchronous multiplexing method of the MPEG (Moving Picture Experts Group) standardized as ISO / IEC13818-2 is used, additional information DM is included in an option field provided in a header of a PES (Packetized Elementary Steam). May be inserted.

  Also, to transmit uncompressed video data and audio data, it is standardized as SMPTE (Society of Motion Picture and Television Engineers) 259M "Television-10-Bit 4: 2: 2 Component and 4fsc Composite Digital Signals-Serial Digital Interface". The SDTI format and the SDTI format standardized as SMPTE305M “Television-Serial Data Transport Interface (SDTI)” for transmitting compressed video data and audio data are further limited. When using the SDTI-CP format standardized as SMPTE 326M “Television-SDTI Content Package Format (SDTI-CP)”, the auxiliary information DM is a UMID standardized as SMPTE 330M “Television-Unique Material Identifier (UMID)”. Data can be inserted into each format signal. Erareru. The method of connecting the attached information DM to the video data DV and the audio data DA is not limited to this, and various methods can be considered.

  The UMID is an identifier uniquely determined for identifying video data, audio data, and other material data. In the UMID, a Basic UMID or an Extended UMID obtained by adding signature metadata (SignatureMetadata) to the Basic UMID is defined. The Basic UMID stores a label for identifying digital data, information indicating whether or not overwriting processing or editing processing has been performed on material data, a number for distinguishing material data, and the like. The signature metadata includes information on the time and date when the material data was generated, correction information (time difference information) on the time when the material data was generated, position information represented by latitude, longitude and altitude, organization name Information can be stored. Furthermore, a user code area is provided in the signature metadata. For example, the additional information DM is stored in this area.

  The Extended UMID generated in this manner is inserted into a signal of each format. Here, when the SDI format is used, an Extended UMID is inserted into an ancillary data area as shown in FIG. When the SDTI format is used, Extended UMID is inserted into an ancillary data (Ancillary Data) area excluding a header data portion as shown in FIG. In the SDTI-CP format, data to be inserted into the payload area is inserted in “item” units. Specifically, as shown in FIG. 6, a system item including information on images and sounds, a picture item including video data, an audio item including audio data, and other data. AUX items are inserted in this order. In the system item, areas of “Package MetadataSet”, “Picture Metadata Set”, “Audio Metadata Set”, and “Auxiliary Metadata Set” are provided so that metadata can be inserted. Therefore, an Extended UMID can be inserted into these areas.

  By the way, the above-described imaging device 10 generates the material data DTm of the desired set frame rate FRs by changing the timing of reading out the imaging charge in the imaging unit 12, and continuously changes the set frame rate FRs. it can. However, when it is only necessary to change the set frame rate FRs in a stepwise manner, the frame data is thinned out, so that the material data DTm of the desired set frame rate FRs can be generated. That is, video data DVa having a higher frame rate than the set frame rate FRs and having a constant frame rate is generated, and video data is extracted from the video data DVa by the set frame rate FRs, whereby the set frame rate is obtained. Video data DV of FRs can be generated. FIG. 7 shows the configuration in this case. In FIG. 7, parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and detailed description is omitted.

  The timing generator 182 of the control unit 18 generates a drive signal CRa corresponding to the maximum value of the set frame rate FRs set via the user interface unit 16 and supplies the drive signal CRa to the imaging unit 12. The imaging unit 12 generates an imaging signal based on the drive signal CRa, generates an imaging signal Spa having a fixed frame rate FRq whose frame rate is higher than the reference frame rate FRr, and sends the imaging signal Spa to the camera processing circuit 131 of the signal processing unit 17. Supply. For example, when the set frame rate FRs can be changed up to n times (n is a positive number) times the reference frame rate FRr, the imaging signal Spa having a frame rate n times the reference frame rate FRr is generated, and the camera processing circuit is generated. 131. That is, the imaging unit 12 generates the imaging signal Spa having a fixed frame rate without being affected by the set frame rate FRs set via the user interface unit 16.

  Further, the timing generator 182 generates a timing signal CTa synchronized with the drive signal CRa and supplies the timing signal CTa to the camera processing circuit 131, the audio processing circuit 132, and the effective frame signal generation circuit 183 of the signal processing unit 17.

  The camera processing circuit 131 supplies the effective data selection circuit 171 with the video data DVa having the fixed frame rate FRq generated based on the imaging signal Spa. The audio processing circuit 132 supplies the valid data selection circuit 171 with audio data DAa generated by performing sampling based on the constant frequency timing signal CTa.

  The imaging control circuit 181 generates a setting information signal CF indicating the setting frame rate FRs based on the operation signal PSa from the user interface unit 16 and supplies the setting information signal CF to the effective frame signal generation circuit 183.

  The effective frame signal generation circuit 183 extracts data from the video data DVa on a frame basis based on the ratio between the frame rate FRq, which is a constant value of the video data DVa, and the set frame rate FRs indicated by the setting information signal CF. An extraction control signal CC for generating video data DV of the set frame rate FRs is generated. Further, the valid frame signal generation circuit 183 supplies the extraction control signal CC to the valid data selection circuit 171 in synchronization with the timing signal CTa. For example, when the frame rate FRq of the video data DVa is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, every other frame from the video data DVa becomes a frame unit. Generates an extraction control signal CC for performing data extraction, and supplies it to the valid data selection circuit 171 in synchronization with the timing signal CTa.

  Further, the valid frame signal generation circuit 183 generates frame rate information DM-FRs indicating the set frame rate FRs based on the setting information signal CF, and supplies the frame rate information DM-FRs to the output unit 15. Further, since the number of frames during the frame period of the reference frame rate FRr can be identified by the extraction control signal CC, the subframe number BN for the frame during each frame period of the reference frame rate FRr is set, and the subframe number BN is also set. It is supplied to the output unit 15 as frame identification information DM-BN.

  The valid data selection circuit 171 extracts the video data DVa and the audio data DAa of the frame indicated by the extraction control signal CC and supplies them to the output unit 15 as the video data DV and the audio data DA. Although not shown, the effective frame signal generation circuit 183 supplies the effective data selection circuit 171 with frame rate information DM-FR indicating the set frame rate FRs. The thinning of the audio data DAa may be performed in accordance with the ratio to the frame rate when the audio data DAa is generated. For example, when the frame rate FRq when the audio data DAa is generated is n times the reference frame rate FRr and the set frame rate FRs is (n / 2) times the reference frame rate FRr, the audio data DAa Thinning out every other sample. In this case, since the thinning interval can be made smaller than in the case where the audio data is thinned in frame units, the sound based on the audio data DA can have good sound quality.

  As described above, by making the frame frequency of the video data DVa constant, it is not necessary to change the operation frequency of the imaging unit 12 or the signal processing unit 17 in the camera processing circuit 131. The configuration can be simplified. Further, since the video data DV of the set frame rate FRs can be generated simply by performing data extraction on a frame basis from the video data DVa, the video data DV of the desired set frame rate FRs can be easily generated from the video data DVa.

  Further, the image data, the adder, and the divider may be provided in the imaging apparatus, and the video data may be added for each predetermined frame to generate the video data DV. In this case, the variable range of the frame rate of the imaging signal Sp can be narrowed, and the set frame rate FRs can be continuously varied. That is, if the image signal Sp for n frames is added and the signal level is multiplied by (1 / n), the frame rate of the image signal Sp can be increased to (1 / n) without increasing the frame rate of (1 / n). A doubled signal can be obtained. Further, by continuously varying the timing of reading out the imaging charges, the set frame rate FRs can be varied continuously.

  FIG. 8 and FIG. 9 are diagrams for explaining the relationship between the video data DV generated by the imaging devices 10 and 10a and the attached information DM. Assuming that the set frame rate FRs is, for example, one or two times the reference frame rate FRr as shown in FIG. 8A, the video data DV shown in FIG. 8B (in the figure, a frame image based on the video data DV is shown). Then, the attached information DM including the frame rate information DM-FRs shown in FIG. 8C indicating the set frame rate FRs and the frame identification information DM-BN shown in FIG. 8D indicating the subframe number BN is connected. FIG. 8E shows the relationship between the passage of time and the frame image. Further, the frame rate information DM-FRs may indicate not only the set frame rate FRs but also a magnification of the set frame rate FRs with respect to the reference frame rate FRr. The magnification is shown in the frame rate information DM-FRs shown in FIG. 8C and the following figures.

  Assuming that the set frame rate FRs is, for example, one time or half the reference frame rate FRr as shown in FIG. 9A, the video data DV shown in FIG. 9B (in the figure, a frame image based on the video data DV is shown). 9C is connected to the frame rate information DM-FRs shown in FIG. 9C indicating the set frame rate FRs and the frame identification information DM-BN shown in FIG. 9D indicating the subframe number BN. FIG. 9E shows the relationship between the passage of time and the frame image.

  FIG. 10 shows a configuration of the signal recording device 20, for example, a configuration of a video tape recorder.

  When material data DTm in which additional information including frame rate information of the main data is connected to main data indicating an image and / or sound is input, the material data DTm is used as recording data by the signal recording processing unit 21. It is supplied to the encoder 211. The encoder 211 uses the supplied material data DTm to generate an error correction code, shuffle, multiplex, and channel encode the data based on an operation control signal CTM supplied from a recording / reproduction control unit 24 described later. To generate a recording signal SW. Further, the generated recording signal SW is supplied to the terminal a of the changeover switch 212.

  The changeover switch 212 is supplied with a changeover control signal CTW from the recording / reproduction control unit 24. The operation of the changeover switch 212 is controlled based on the changeover control signal CTW, and the recording signal SW is connected to the terminal b. The signal is supplied to the amplifier 213a or the recording amplifier 213b connected to the terminal c.

  The recording amplifier 213a amplifies the supplied recording signal SW and supplies the signal SW to the signal switch 221a of the signal recording / reproducing unit 22. The recording amplifier 213b amplifies the supplied recording signal SW and supplies the amplified recording signal SW to the signal switch 221b.

  The signal switch 221a supplies the signal supplied from the recording amplifier 213a to the magnetic head 222a at the time of signal recording, based on the switching control signal CWR from the recording / reproduction controller 24. At the time of signal reproduction, the signal obtained by the magnetic head 222a is supplied to the reproduction amplifier circuit 231a of the signal reproduction processing unit 23. Similarly, the signal switch 221b supplies the signal supplied from the recording amplifier 213b to the magnetic head 222b at the time of signal recording based on the switching control signal CWR from the recording / reproduction controller 24. At the time of signal reproduction, the signal obtained by the magnetic head 222b is supplied to the reproduction amplification circuit 231b of the signal reproduction processing unit 23.

  In this way, by supplying the amplified recording signal SW to the magnetic heads 222a and 222b provided on the rotating drum (not shown), images and sounds and images are recorded on a magnetic tape (not shown). Record the attached information. The signals recorded on the magnetic tape are read by the magnetic heads 222a and 222b and supplied to the reproduction amplifier circuits 231a and 231b.

  The reproduction amplifier circuit 231a amplifies the signal obtained by the magnetic head 222a and supplies the amplified signal to the terminal a of the changeover switch 232. Further, the reproduction amplifier circuit 231b amplifies the signal obtained by the magnetic head 222b and supplies the signal to the terminal b of the changeover switch 232. The terminal c of the changeover switch 232 is connected to the decoder 233.

  The changeover switch 232 is supplied with a changeover control signal CTR from the recording / reproduction control unit 24. The operation of the changeover switch 232 is controlled based on the changeover control signal CTR, and the signals output from the reproduction amplification circuits 231a and 231b are output. Is selected and supplied to the decoder 233 as the reproduction signal SR. The decoder 233 performs decoding of the reproduction signal SR, data separation, deshuffling, error correction processing, and the like based on the operation control signal CTM supplied from the recording / reproduction control unit 24, and converts the main data representing an image and / or sound into main data. , And generates and outputs material data DTm to which additional information including the frame rate information of the main data is linked.

  A user interface unit 25 is connected to the recording / reproduction control unit 24. Based on an operation signal PSv supplied from the user interface unit 25 and an operation signal PSw supplied from an external device such as the editing device 30, a switching control signal is output. An operation control signal CTM for controlling the operations of the CTW, CTR, encoder 211 and decoder 233 is generated. Further, the recording / reproduction control unit 24 also controls the drive of the magnetic tape and the rotary head. Further, the recording / reproduction control unit 24 displays the operation state of the video tape recorder and various information by generating the display signal PH and supplying the display signal PH to the display unit 26.

  Although FIG. 10 shows the case where the material data DTm is recorded on a magnetic tape, a signal reproducing device using a disk-shaped recording medium using light or magnetism or a semiconductor storage element may be used.

  Further, the signal recording device 20 only needs to be able to connect and record or output the video data DV or the audio data DA and the accessory information DM. The input material data DTm is a material in which the accessory information is inserted into a data stream, a header, or the like. It is not limited to the data DTm. For example, they may be supplied via different signal lines.

  As described above, if the auxiliary information DM is connected to the main data, editing processing or data editing can be performed using data generated by the imaging devices 10 and 10a or data recorded on the recording medium by the signal recording device 20. Variable speed reproduction can be easily performed.

  Here, in the editing processing of the data in which the attached information DM is linked to the main data, the reproducible speed is set based on the set frame rate FRs. In the setting of the reproducible speed, the reproducible speed is set so as to obtain a natural reconstructed image by thinning out at a fixed frame interval or repeating the frame. For example, when the multiple of the set frame rate FRs with respect to the reference frame rate FRr is larger than 1, that is, when FRs / FRr> 1, a divisor of this multiple other than 1 is obtained, and the reciprocal of the obtained divisor is obtained. Is calculated, the reproducible speed can be set. That is, when the set frame rate FRs is ten times the reference frame rate, divisors other than 1 are "2, 5, 10", and the reciprocal of this divisor "1/10, 1/5, 1/2" It becomes. By selecting the reciprocal of this divisor, a reproducible speed lower than 1 × speed can be set. Further, by setting the reproducible speed equal to or higher than 1 × speed to a natural number multiple, the thinning intervals performed at each reproduction speed can be made equal.

  When the multiple of the set frame rate FRs with respect to the reference frame rate FRr is smaller than 1, that is, when FRs / FRr <1, the reciprocal of this multiple is obtained and the value of a divisor other than 1 or an integer multiple of the reciprocal is calculated. By calculating, the reproducible speed can be set. That is, when the set frame rate FRs is (１ ／) times the reference frame rate, the divisor other than 1 is “2, 3, 6”, and the integral multiple of the reciprocal is “6, 12, 18, ... ". If a value among these values is selected as the reproducible speed, the thinning interval or the number of repetitions performed at each reproduction speed can be made equal. By setting the reproducible speed equal to or lower than 1 × speed to (1 / natural number) times, the number of repetitions at each reproduction speed can be made equal. However, since the image is repeated, the lower limit of the reproducible speed can be set to 1 × speed. good.

  FIG. 11 shows a GUI screen for an editing operation using the attached information. For example, on the GUI screen, a material management browser 401 is provided at the upper left as a material management display, and a storyboard 402 is provided at the lower left as a playback order display. At the center of the screen, a monitor viewer 403 for displaying the images before and after editing is provided as a playback image display, and at the lower center of the screen, a timeline 404 is displayed as a playback time order display. An operation control unit 405 is provided between the monitor viewer 403 and the timeline 404. Note that FIG. 11A shows a GUI screen when the reproduction speed is 1 ×, FIG. 11B shows a (1/10) × speed, and FIG. 11C shows a GUI screen when the reproduction speed is 2 ×. In addition, the arrangement and shape of the material management browser 401, storyboard 402, monitor viewer 403, timeline 404, and operation control unit 405 are merely examples and are not limited.

  The material management browser 401 displays a list of material data used for editing, and displays a title image, a length and a stamp image (thumbnail image) indicating the content of the stored material data for each material.

  The storyboard 402 is a place where content generation is performed, and generates content by arranging material data in the order of reproduction. The monitor viewer 403 not only displays an image based on the material data, but also displays a reproduction position and a variable speed bar indicating a variable range of the reproduction speed.

  The timeline 404 is a place for performing content generation work in more detail by pasting material data along the time axis. The operation control unit 405 displays operation keys for reproducing material data and contents arranged on the timeline 404.

  Here, when the selected material data is reproduced, the set frame rate FRs is identified based on the attached information DM, and the reproduction speed FP is multiplied by the set frame rate FRs to calculate an identification value FD. The reproduction processing condition is determined based on. For example, as shown in the screen display of FIG. 11A, when the reproduction speed is 1 × speed and the set frame rate FRs is 10 × speed with respect to the reference frame rate FRr, the identification value FD is “10 × 1 = 10 ". FIG. 12A shows an image based on the video data DV when the set frame rate FRs is set to be 10 times faster than the reference frame rate FRr. 12B shows frame rate information DM-FRs indicating a set frame rate FRs of a frame image, FIG. 12C shows frame identification information DM-BN showing a subframe number BN, and FIG. 12D shows an absolute frame number AN. Here, since the set frame rate FRs is set to be 10 times faster than the reference frame rate FRr, one frame period of the reference frame rate FRr includes 10 frame images of the set frame rate FRs. , The subframe number BN is repeated from “0” to “9”.

  When the identification value FD is “FD = 10”, as shown in FIGS. 12E to 12G, the video signal Svm is generated by using the video data DV every tenth frame, that is, while skipping the video data DV by nine frames. A 1 × speed reproduced image can be displayed on the monitor viewer 403 based on the signal Svm. 12E shows frame identification information DM-BN indicating the sub-frame number BN of the displayed image, FIG. 12F shows the absolute frame number AN of the displayed image, and FIG. 12G shows the frame image displayed by the video signal Svm. ing.

  Next, when an operation of expanding the display width of the frame image display area 404a in the timeline 404 in the direction of arrow A or an operation of moving the cursor position indicated by the thick line of the speed-variable console display 403c in the direction of arrow B is performed, the playback speed is increased. Is treated as having been operated to slow down. Here, for example, when the speed is set to (1/10) speed, the reproduction speed display 403b of the monitor viewer 403 is changed to (1/10) speed as shown in FIG. 11B. The cursor position on the speed-variable console display 403c is set to a position at (1/10) times the speed. Since the reproduction time is further increased, the display width of the frame image display area 404a is increased.

  As shown in the screen display of FIG. 11B, in reproduction at a reproduction speed of (1/10), the identification value FD becomes “10 × (1/10) = 1”. When the identification value FD is “FD = 1”, a video signal Svm is generated using every “FD = 1” frame, that is, using the video data DV for each frame, and based on the video signal Svm, as shown in FIG. 12A. A reproduced image at (1/10) speed can be displayed on the monitor viewer 403.

  Further, when an operation of reducing the display width of the frame image display area 404a on the timeline 404 in the direction of arrow B or an operation of moving the cursor position of the speed-variable console display 403c in the direction of arrow A is performed, an operation of increasing the playback speed is performed. Treat it as having been done. Here, for example, when the double speed is set, the reproduction speed display 403b of the monitor viewer 403 is changed to the double speed as shown in FIG. 11C. The cursor position on the variable speed console display 403c is set to a double speed position. Since the reproduction time is further shortened, the display width of the frame image display area 404a is reduced.

  As shown in the screen display of FIG. 11C, in the case of the reproduction at the double speed, the identification value FD is “10 × 2 = 20”. When the identification value FD is “FD = 20”, as shown in FIGS. 12H to 12K, the video signal Svm is generated using every “FD = 20” frame, that is, while skipping the video data DV by 19 frames. Thus, a double-speed playback image can be displayed on the monitor viewer 403 based on the video signal Svm. 12H shows the frame identification information DM-BN indicating the sub-frame number BN of the displayed image, FIG. 12J shows the absolute frame number AN of the displayed image, and FIG. 12K shows the frame image displayed by the video signal Svm. ing.

  As described above, by using the attached information DM, it is possible to easily generate desired content data for broadcast or distribution by combining the material data of the set frame rate FRs with various playback speeds. Also, if the content data is generated by linking the additional information corresponding to the edited video data, by performing the same processing as the editing apparatus on the user side using the additional information included in the content data, It is also possible for the user to change the playback speed of the content. For example, by setting the set frame rate FRs higher than the reference frame rate FRr, material data for sports broadcasting or the like is generated, and using this material data, Is generated, the user can normally view only a desired scene at a slow speed while watching at 1 × speed.

  As described above, the present invention is useful when the frame rate of video data is variably set to a desired set frame rate, and the additional information including the frame rate information is linked to the video data. This is suitable when editing processing is performed using video data whose rate is variably set as a material.

FIG. 1 is a diagram illustrating a configuration of a content providing system. FIG. 2 is a diagram illustrating a configuration of an imaging device. 9 is a flowchart illustrating an operation of adding a subframe number. It is a figure showing an SDI format. It is a figure showing an SDTI format. It is a figure which shows SDTI-CP format. FIG. 11 is a diagram illustrating another configuration of the imaging device. FIG. 6 is a diagram illustrating a relationship (part 1) between video data and attached information. It is a figure which shows the relationship (2) of video data and attached information. FIG. 2 is a diagram illustrating a configuration of a video tape recorder. FIG. 14 is a diagram illustrating a GUI screen at the time of an editing operation. It is a figure showing an image reproduction operation.

Explanation of reference numerals

10, 10a: imaging apparatus, 12: imaging section, 13, 17: signal processing section, 14, 18: control section, 15: output section, 16: user interface section, 19: voice input device, 131: camera processing circuit, 132: voice processing circuit, 141, 181: imaging control circuit, 142, 182: timing generator, 171: valid data selection Circuit, 183: Effective frame signal generation circuit, 401: Material management browser, 402: Storyboard, 403: Monitor viewer, 403b: Playback speed display, 403c: Speed variable console display , 404... Time line, 404 a, frame image display area, 405, operation control unit

Claims (14)

An imaging unit that reads a signal from an imaging element and generates an imaging signal based on a captured image;
Signal processing means for generating video data based on the imaging signal,
Control means for controlling the operation of at least one of the image pickup means and the signal processing means, variably setting a frame rate of the video data to a desired set frame rate, and generating at least additional information indicating the set frame rate; ,
An output unit that outputs the video data by linking the attached information to the video data. 2. The imaging apparatus according to claim 1, wherein the control unit variably sets a frame rate of the video data by changing a timing at which a signal is read from the imaging element. 3. 2. The imaging apparatus according to claim 1, wherein the control unit variably sets a frame rate of the video data by performing frame thinning by controlling the signal processing unit. The control unit variably controls a signal readout timing from the image sensor, controls the signal processing unit to perform frame addition, and variably sets a frame rate of the video data. The imaging device according to 1. The control means adds a subframe number to each frame of the set frame rate included in one frame period of a reference frame rate, and includes the subframe number in the additional information. The imaging device according to 1. The signal processing means generates audio data by sampling an analog audio signal,
The control unit controls a sampling period of the analog audio signal in the signal processing unit based on the set frame rate, and the output unit outputs the video data and the audio data by linking the attached information. The imaging device according to claim 1, wherein: The imaging apparatus according to claim 1, wherein the output unit outputs the connected signal by recording a signal obtained by connecting the attached information to the video data on a recording medium. Generating video data based on a signal read from the image sensor;
Variably setting a frame rate of the video data to a desired set frame rate;
Generating ancillary information including frame rate information indicating the set frame rate,
An image pickup method, comprising a step of connecting and outputting the additional information to the video data. 9. The imaging method according to claim 8, wherein in the step of variably setting, a frame rate of the video data is variably set by variably setting a signal readout timing from the image sensor. 9. The imaging method according to claim 8, wherein in the step of variably setting, the frame rate of the video data is variably set by performing frame thinning. 9. The variably setting step variably sets a signal readout timing from the image sensor and performs frame addition to variably set a frame rate of the video data. Imaging method. The imaging method according to claim 8, wherein the additional information includes a subframe number assigned to each frame of the set frame rate included in one frame period of a reference frame rate. Generating audio data by sampling an analog audio signal, and further comprising controlling a sampling period of the analog audio signal based on the set frame rate,
9. The imaging method according to claim 8, wherein, in the outputting, the additional information is connected to the video data and the audio data and output. 9. The imaging method according to claim 8, wherein, in the outputting step, a signal in which the additional information is linked to the video data is recorded on a recording medium, so that the linked signal is output. .

Images