JP4760893B2 - Movie recording / playback device - Google Patents

Description

  The present invention relates to a moving image recording / reproducing apparatus for recording and reproducing moving image data.

In recent years, it has become possible to acquire or view a large amount of moving image data by using multi-channel broadcasting of moving image data by digital television broadcasting or widening the network. In addition, improvements in video compression / decompression technology, lower hardware / software costs, and higher storage media capacity and lower prices make it easier to store large amounts of video data. The video data that can be recorded and viewed is increasing. However, for busy people, there is no time to view all the moving image data, and as a result, the viewable moving image data is flooded. Therefore, it is important to view only the important scenes in the moving image data so that the mechanism for understanding the contents in a short time or the moving image data that the user really wants to view can be found.

In view of such a situation, for example, Patent Document 1 and Non-Patent Document 1 introduce a technique capable of displaying an important scene in moving image data. In particular, Non-Patent Document 1 generates a feature from video data, extracts and ranks important scenes based on the features, and reproduces only the important scenes at a rate specified by the user. Has been introduced.
With these technologies, the user can grasp the contents of the moving image data in a short time.

JP 2003-153139 A D. DeMenthon, V. Kobla, and D. Doermann, Tenideo Summarization by Curve Simplification ACM Multimedia 98, Bristol, England, pp.211-218, 1998

As described above, a technique for grasping the content of video data in a short time by displaying only important scenes (referred to as highlight playback in the following description and drawings) has been proposed. The definition of a simple scene is not unique to all moving image data, and varies depending on the category of moving image data. For example, when the moving image data is a drama, the importance of a long scene should be increased, and when it is a sport, the importance of a scene with a high sound power should be increased. Even in sports, it is desirable to change the length of the scene to be played back, including the position where the sound power is increased. That is, it is necessary to change the extraction method and the ranking method of important scenes depending on the category of moving image data. If this is not done, it is not possible to perform suitable highlight reproduction for all moving image data.

The present invention solves these problems and provides a moving image recording / reproducing apparatus capable of performing optimum highlight reproduction according to the category of moving image data.

In order to solve the above-described problem, the moving image recording / reproducing apparatus of the present invention includes a storage unit that stores moving image data received as broadcast data or acquired via a network, and a reproducing unit that reproduces moving image data. A feature data holding unit that holds video or audio features for each frame constituting moving image data stored in the storage unit, a category acquisition unit that acquires a category of moving image data stored in the storage unit, and a feature data holding unit A scene including one or more frames that are candidates for highlight reproduction and its ranking determined by an algorithm according to the category acquired by the category acquisition unit based on the feature data for each frame held in The ranking data holding unit and the highlight playback instruction including the specification of playback conditions from the outside, the ranking A playback scene determining unit that determines whether or not each scene held in the data holding unit should be played back based on the designation of the playback condition and the ranking, and the playback unit is determined by the playback scene determining unit The two or more playback scenes are played back in order of playback time.

Since the highlight reproduction can be appropriately performed on the moving image data of all categories, it is possible to appropriately and efficiently grasp the contents of the viewable moving image data that is overflowing.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is an example of a hardware configuration of a moving image recording / playback apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the moving picture recording / reproducing apparatus according to the first embodiment includes a central processing unit 101, an input device 102, a display device 103, an audio output device 104, a storage device 105, and a secondary storage device 106. It is comprised. Further, when acquiring moving image data and an electronic program guide via a network, the data transmission / reception device 108 is further provided. When receiving broadcast data and obtaining moving image data and an electronic program guide, the broadcast data input device 100 is further included. Each device is connected by a bus 107 and configured so that data can be transmitted and received between the devices.

The central processing unit 101 is mainly composed of a microprocessor, and a storage device 105.
And a program stored in the secondary storage device 106 is executed.

The input device 102 is realized by, for example, a remote controller or a pointing device such as a keyboard or a mouse, and the user of the moving image recording / playback device performs operation settings related to moving image recording and playback through the input device 102. In addition, the user can specify the category of the moving image data through the input device 102.

The display device 103 is realized by, for example, a display adapter, a liquid crystal panel, a projector, and the like, and displays a reproduced image of moving image data by the moving image reproducing device.

The audio output device 104 is realized by a speaker, for example, and outputs audio included in the moving image data.

The storage device 105 is, for example, a random access memory (RAM) or a read only memory (ROM).
And the like, a program executed by the central processing unit 101, data processed by the moving image recording / playback device, moving image data, and the like are stored.

The secondary storage device 106 is, for example, a hard disk, DVD or CD and their drives,
Alternatively, it is configured by a non-volatile memory such as a flash memory, and stores a program executed by the central processing unit 101, data processed by the moving image recording / reproducing device, moving image data, and the like.

The broadcast data input device 100 is realized by a TV tuner or the like, and enables input of moving image data broadcast on the TV or the like. Further, when an electronic program guide is acquired from broadcast data, the data of the electronic program guide can be acquired.

The network data transmission / reception device 108 is realized by a network card such as a LAN card, and enables input of moving image data and electronic program guide data from other devices connected via the network.

FIG. 2 is a functional block diagram of the moving image recording / playback apparatus according to the first embodiment of the present invention. Some or all of these functional blocks may be realized as hardware in addition to the hardware shown in FIG. 1, or may be realized as a software program executed by the central processing unit 101. In the following description, it is assumed that all of these functional blocks are software programs executed by the central processing unit 101 as an example.

As shown in FIG. 2, the moving image recording / playback apparatus according to the first embodiment includes a recording block 201, a feature generation block 202, a ranking generation block 203, and a playback block 204.

The recording block 201 is a block for recording moving image data input from the outside, and includes a recorded moving image data input unit 211, a recording unit 212, a recording unit 213, a category acquisition unit 214, a category table generation unit 215, a category A data holding unit 216 is included.

The recorded moving image data input unit 211 is activated by the recording unit 212, and the broadcast data input device 1.
00 or moving image data to be recorded is input from the network data transmitting / receiving device 108. When moving image data is input from the broadcast data input device 100, the moving image data broadcast from a designated channel is acquired in accordance with an instruction from the recording unit 212 described later, converted into digital data, and stored in the storage device 105. To store. On the other hand, when moving image data is input from the network data transmitting / receiving device 108, the moving image data is acquired from the designated device via the network and stored in the storage device 105 in accordance with an instruction from the recording unit 212 described later. At this time, as a method of acquiring the moving image data, the already known HTTP or FTP may be used.

The recording unit 212 starts when recording is instructed by the user via the input device 102 and receives moving image data from the broadcast data input device 100 or the network data transmitting / receiving device 108 via the recorded moving image data input unit 211. The data is input and stored in the secondary storage device 106 by the recording unit 213 described later. At this time, when moving image data is input from the broadcast data input device 100, a preset channel or a channel designated by the user is set in the broadcast data input device 100 via the recorded moving image data input unit 211. And
The moving image data of the channel is input and stored in the secondary storage device 106 by the recording unit 213. At this time, the input moving image data is compressed by a method such as MPEG and the secondary storage device 106.
May be stored. On the other hand, when moving image data is input from the network data transmitting / receiving device 108, a preset URI or a URI designated by the user is set in the network data transmitting / receiving device 108 via the recorded moving image data input unit 211. , The URI
The moving image data is input and stored in the secondary storage device 106 by the recording unit 213. Also, a category table in which the category (type of drama, baseball, soccer, etc.) of the input moving image data is acquired by the category acquiring unit 214 described later and associated with the recorded moving image data recorded in the secondary storage device 106 by the category table generating unit 215. And the generated category table is held in the category data holding unit 216.

The recording unit 213 stores the input moving image data from the recorded moving image data input unit 211 in the secondary storage device 106 according to an instruction from the recording unit 212. At this time, a unique file name is assigned to the recorded moving image data.

In response to an instruction from the recording unit 212, the category acquisition unit 214 records the recorded moving image data input unit 21.
The category of input moving image data from 1 is acquired. For example, if the electronic program guide data including information related to the video data has already been acquired, the category of the video data is acquired from the data of the electronic program guide, and the data of the electronic program guide is acquired. If not, the electronic program guide data including information related to the moving image data is newly acquired, and then the category of the moving image data is acquired from the electronic program guide data. Note that the electronic program guide data is obtained by using the broadcast data input device 100 or the network data transmitting / receiving device 108 as the electronic program guide data included in the broadcast data or the electronic program guide data provided by another server via the network. What is necessary is just to comprise so that it may acquire and memorize | store in the memory | storage device 105 or the secondary memory | storage device 106. FIG. The data structure of the electronic program guide will be described later. Specifically, when moving image data is input from the broadcast data input device 100, the data of the electronic program guide corresponding to the time and channel is stored in the storage device 105 or the secondary from the information on the time and channel when recording started. It is determined whether or not it is stored in the storage device 106, and if it is already stored, the corresponding category can be obtained from the electronic program guide data using the recording start time and channel information as keys. Good.

On the other hand, if not yet stored, electronic program guide data broadcast on a predetermined channel is acquired via the broadcast data input device 100, or predetermined data via the network data transmitting / receiving device 108. The electronic program guide data is obtained from the URI, and the corresponding category is obtained from the electronic program guide data using the time of recording start and channel information as keys. That is, when the electronic program guide data including information on the input moving image data has not been acquired yet, it means that the electronic program guide data is acquired with the start of recording as a trigger. Or this category acquisition part 21
4 may use a category designated by the user as a category of the input moving image data. Also,
Even when moving image data is acquired on the network via the network data transmitting / receiving device 108, in the case of data being broadcast, the electronic program guide data or the category specified by the user is input as described above. It may be a data category, or when the moving image data category is included in a predetermined URI or moving image data URI, these data may be acquired and used as the category.

The category table generation unit 215 is activated by the recording unit 212 and stores the moving image data stored in the secondary storage device 106 by the recording unit 213, that is, the recorded moving image data and its category, that is, the input moving image data acquired by the category acquisition unit 214. Associate categories
A category table exemplified later is generated and stored in the storage device 105 or the secondary storage device 106 via the category data storage unit 216. The category table generation unit 215
And the category table is for making it possible to refer to the category of the recorded moving image data at any time after the end of recording, and at the same time as recording or continuing after the end of recording, When the recorded video data and the category are associated with each other, such as generating a ranking, the category acquired by the category acquisition unit 214 may be directly held by the category data holding unit 216 described later. Is
The category table generation unit 215 and the category table are not necessarily required. The category table generation unit 215 may associate the recorded moving image data with the category by a method such as making the file name of the recorded moving image data the same as the file name storing the category data.
In this case, the category table is not necessarily required.

The category data holding unit 216 stores the category table generated by the category table generation unit 215 in the storage device 105 or the secondary storage device 106 according to an instruction from the recording unit 212. As described above, when the recorded video data and the category are associated with each other, such as generating feature data and ranking of the video data being recorded or recorded at the same time as recording or after the end of recording, The category data holding unit 216 directly stores the category acquired by the category acquisition unit 214 as the storage device 105 or the secondary storage device 106.
You may comprise so that it may memorize | store. When the category table generating unit 215 associates the recorded moving image data with the category by, for example, making the file name of the recorded moving image data the same as the file name storing the category data, the category data holding unit 216 The category data of the moving image data having the same file name as the recorded moving image data, that is, the category data acquired by the category acquisition unit 214 may be stored in the storage device 105 or the secondary storage device 106. Good.

Next, the feature data generation block 202 will be described. Feature data generation block 2
Reference numeral 02 denotes a block that generates features of input moving image data or moving image data to be reproduced, and includes an analysis moving image data input unit 221, a feature generation unit 222, and a feature data holding unit 223.

The analysis moving image data input unit 221 inputs moving image data targeted for highlight reproduction from the storage device 105 or the secondary storage device 106. At this time, the file name of the moving image data is acquired. The analysis moving image data input unit 221 is used after recording starts, after recording ends, when the user instructs creation of feature data and ranking data, or when playback starts.
Alternatively, it is activated when moving image data for which feature data and ranking data are not created is found by a scheduler (not shown). Also, the analysis video data input unit 221
May be activated when recording is started, and in this case, the recording unit 212 may be configured to input the moving image data being input or being recorded. In this case, the file name of the moving image data being recorded is acquired.

The feature generation unit 222 generates the feature of the moving image data input in the analysis moving image data input unit 221. This can be realized, for example, by generating audio power, image motion, and the like for each frame of audio data and image data in moving image data. Note that the feature generation unit 222 is executed each time moving image data is input when the analysis moving image data input unit 221 is executed.

The feature data holding unit 223 holds the feature data generated by the feature generation unit 222. This can be realized, for example, by storing the feature data generated by the feature generation unit 222 in the storage device 105 or the secondary storage device 106. At this time, the feature data to be held is given the same file name as the file name of the recorded moving image data acquired by the analysis moving image data input unit 221. The feature data holding unit 223 is configured to be executed every time feature data is generated or every time one frame of feature data is generated when the feature generation unit 222 is executed. That's fine.

Next, the ranking generation block 203 will be described. Ranking generation block 2
03 is a block that performs ranking according to importance for each scene in the input moving image data or the reproduction target moving image data. The feature data input unit 231, the category data input unit 232, the ranking generation unit 233, the ranking A data holding unit 234 and ranking generation units 235 to 237 for each category of moving image data are included.

The feature data input unit 231 inputs the feature data held in the feature data holding unit 223. This can be realized, for example, by reading feature data stored in the storage device 105 or the secondary storage device 106. At this time, the feature data file name is acquired. The feature data input unit 231 may be executed when a ranking generation unit 233 described later is executed.

The category data input unit 232 acquires category data of the recorded moving image data that is a ranking generation target. In the case of this example, since the file name of the recorded moving image data is the same as the file name of the feature data as described above, the category data acquisition unit 216 obtains the category data for the recorded moving image data as described later. 105 or secondary storage device 1
This can be realized by acquiring corresponding category data from the category table held in 06 using the file name of the feature data acquired by the feature data input unit 231 as a key. The category data input unit 232 may be executed when a ranking generation unit 233 described later is executed.

235 to 237 are ranking data generation units according to the category of the moving image data. In particular, as an example, 235, 236, and 237 are the cases where the moving image data category is sumo, soccer, and drama, respectively. The ranking data generation unit is shown. The ranking generation unit corresponding to these categories is activated according to the category of the moving image data by the ranking generation unit 233 described later, and based on the feature data input by the feature data input unit 231, extraction of important scenes from the moving image data And ranking each scene.
For example, when the video data category is sumo, the ranking generation according to each category increases the importance of a predetermined number of scenes from the time when the subtitles of the approach appear, and the audio power In the case of soccer, the importance of several scenes before and after the scene in which the sound power is increased is increased, or in the case of a drama, it is described in Non-Patent Document 1. It is preferable to use an algorithm that is optimal for the category, such as using a method.

The ranking generation unit 233 activates a ranking generation unit corresponding to each category in accordance with the category of the moving image data input by the category data input unit 232, extracts an important scene optimal for the category of moving image data, and displays each scene Ranking. The ranking generation unit 233 generates ranking data at the start of reproduction or by a scheduler (not shown) when the user is instructed to generate ranking data after recording starts, after recording ends, after feature data generation ends. What is necessary is just to comprise so that it may be started when it discovers the video data which is not.

The ranking data holding unit 234 holds the ranking data generated by the ranking generation unit 233. This can be realized, for example, by storing the ranking data generated by the ranking generation unit 233 in the storage device 105 or the secondary storage device 106. The ranking data holding unit 234 may be configured to be executed every time ranking data is generated when the ranking generating unit 233 is executed.

Next, the playback block 204 will be described. The reproduction block 204 is a block for performing highlight reproduction of moving image data to be reproduced, and includes a ranking data input unit 241, a reproduction moving image data input unit 242, a reproduction scene determination parameter input unit 243, and a reproduction scene determination unit 244. A playback unit 245, a display unit 246, and an audio output unit 247.

The ranking data input unit 241 inputs the ranking data held in the ranking data holding unit 234. For example, the storage device 105 or the secondary storage device 10
This can be realized by reading out the ranking data stored in 6. The ranking data input unit 241 may be configured to be activated when a playback scene determination unit 244 described later is executed.

The playback scene determination parameter input unit 243 inputs parameters for determining a playback scene in highlight playback via the input device 102 from the user. Note that this parameter includes, for example, viewing time in highlight reproduction of moving image data. The playback scene determination parameter input unit 205 is executed when highlight playback is executed in the playback unit 206 described later.

The playback scene determination unit 244 determines a playback scene in highlight playback based on the parameters input by the playback scene determination parameter input unit 243 and the ranking data input by the ranking data input unit 241. A method for determining the playback scene will be described later. The playback scene determination unit 244 is executed after the playback scene determination parameter input unit 243 receives a parameter for determining a playback scene in highlight playback.

The reproduction moving image data input unit 242 inputs reproduction target moving image data from the secondary storage device 106. The reproduction video data input unit 242 is activated when the reproduction unit 245 described later acquires video data to be reproduced.

The playback unit 245 inputs the playback scene video data determined by the playback scene determination unit 244 via the playback video data input unit 242, generates a playback image, and displays the playback image on the display unit 246 described later. The main playback unit 245 is executed when highlight playback is instructed by the user.

The display unit 246 displays the playback image generated by the playback unit 245 on the display device 103. The main display unit 246 displays the reproduction image generated by the reproduction unit 245 for each frame on the display device 1.
03. In this case, the main display unit 246 may be configured to be activated each time a playback image for one frame is generated by the playback unit 245.

The audio output unit 247 outputs the reproduction audio generated by the reproduction unit 245 to the audio output device 104.
Output to. The audio output unit 247 can be realized by outputting the reproduced audio generated by the reproducing unit 245 to the audio output device 104 frame by frame. In this case, the audio output unit 247 may be configured to be activated every time the playback unit 245 generates playback audio for one frame.

Next, electronic program guide data acquired by the category acquisition unit 214 will be described.

FIG. 3 shows an example of the data structure of the electronic program guide acquired by the category acquisition unit 214 of the moving image recording / playback apparatus according to the embodiment of the present invention.

In FIG. 3, reference numerals 306 to 309 denote broadcast programs, that is, moving image data. For each moving image data, at least the date 300 on which the moving image data is broadcast,
Channel 301, broadcast start time 302, broadcast end time or broadcast time 303,
A title 304 and a category 305 are included. Therefore, the category acquisition unit 214
Can determine whether electronic program guide data corresponding to the moving image data exists from the date, time and channel information of the recording start. Also, by acquiring the corresponding category from the electronic program guide data using the date, time and channel information as a key for recording, it is possible to acquire the category of moving image data to be recorded.
The data structure of the electronic program guide data is an example. For example, the data structure may be a separate file for each broadcast day, or a separate file for each broadcast channel. Even in such a case, it is possible to check the existence of the electronic program guide data corresponding to the moving image data and to acquire the category using the date, time and channel information as keys.

Next, the data structure of the category table generated by the category table generation unit 215 will be described.

FIG. 4 is an example of the data structure of the category table generated by the category table generating unit 215 of the moving picture recording / reproducing apparatus according to the embodiment of the present invention.

In FIG. 4, reference numerals 403 to 406 denote recorded moving image data. For each moving image data, at least the file name 401 of the recorded moving image data and its category 40 are recorded.
2 is included. Therefore, the category data input unit 232 can acquire the category of the recorded moving image data by acquiring the corresponding category data using the file name of the recorded moving image data as a key. In the moving image recording / playback apparatus according to the present embodiment, the feature data holding unit 223 uses the same file name for the feature data generated by the feature data generation unit 222 and the file name for the recorded moving image data. Then, the feature data input unit 231 acquires the feature data and its file name, and the ranking generation unit 233 acquires them. For this reason, the ranking generation unit 233 notifies the category data input unit 232 of the file name of the feature data, so that the category data input unit 232 uses the file name of the feature data as the category of the recorded video data. Can be acquired.

Next, the data structure of the ranking data generated by the ranking generation unit 233 will be described.

FIG. 5 is an example of the data structure of the ranking data generated by the ranking generation unit 233 of the moving image recording / playback apparatus according to the embodiment of the present invention.

In FIG. 5, reference numerals 504 to 508 denote scenes in the moving image data, and reference numeral 501 denotes the importance or rank in each scene. Here, for convenience, the smaller the numerical value, the higher the rank. Reference numerals 502 and 503 denote a start position and an end position of each scene, and here, for convenience, the start time (second) and end time (second) of each scene are used. The ranking generation unit 223 performs the extraction and ranking of scenes in the moving image data in the ranking generation unit corresponding to the category of the moving image data, acquires the time of the start position and the end position of each scene, and this data structure Should be generated. Note that the start time and end time of each scene can be acquired by acquiring a time stamp corresponding to a frame if the moving image data is an MPEG stream, for example. With this ranking data, the playback scene determination unit 244 can determine the scene to be played back by referring to the rank and playback time of each scene.

  Next, the processing flow of each functional block will be described.

FIG. 6 is a flowchart of processing in the recording block 201 of the moving image recording / playback apparatus according to the first embodiment of the invention.

The main recording block 201 operates in response to an instruction from the recording unit 212 when the user instructs the start of recording via the input device 102. In the recording block 201, as shown in FIG. 6, first, the category acquisition unit 214 acquires the category of the input moving image data (step 60).
1). Next, the category table generating unit 215 associates the input moving image data to be recorded with the category, that is, the category acquired by the category acquiring unit 214 in step 601 to generate a category table (step 602), and the category data holding unit At 216, this category table is held. Subsequently, moving image data to be recorded is input by the recorded moving image data input unit 211 (step 603), and the input moving image data to be recorded is recorded in the secondary storage device 106 by the recording unit 213 (step 604). At this time, a unique file name is assigned to the recorded moving image data. Note that the input moving image data may be compressed by a method such as MPEG and stored in the secondary storage device 106.

FIG. 7 is a flowchart of processing in the feature generation block 202 of the moving image recording / playback apparatus according to the embodiment of the present invention.

This feature generation block 202 generates feature data and ranking data after recording starts, after recording ends, or when the user instructs creation of feature data and ranking data, or when playback is started, or by a scheduler not shown. It is activated when video data that has not been recorded is found. As shown in FIG. 7, the feature generation block 202 first acquires video data that has been recorded or is being recorded in the analysis video data input unit 221 (step 701). At this time, the file name of the moving image data is acquired. Subsequently, the feature generation unit 222 generates the feature of the moving image data input in step 701 (step 702), and the generated feature data is held in the feature data holding unit 223.

FIG. 8 is a flowchart of processing in the ranking generation block 203 of the moving image recording / playback apparatus according to the embodiment of the present invention.

This ranking generation block 203 is used when the user is instructed to create ranking data after the start of recording, after the end of recording, after the end of feature data generation, by the instruction of the ranking generation unit 233, at the start of reproduction, or a scheduler (not shown). Is activated when it finds video data for which ranking data has not been created. This ranking generation block 2
03, as shown in FIG. 8, first, feature data is input by the feature data input unit 231 (step 801). At this time, the feature data file name is acquired. Subsequently, the category data input unit 232 acquires category data of the recorded moving image data to be ranked (step 802). In this example, since the file name of the recorded moving image data and the file name of the feature data are the same as described above, the category data for the recorded movie data is acquired by the file name of the feature data acquired in step 801. By acquiring corresponding category data using as a key, it is possible to acquire category data of recorded moving image data to be ranked. Next, start the ranking generation unit according to the category,
Extract important scenes from video data and rank each scene (Step 8)
03) The generated ranking data is held in the ranking data holding unit 234.

FIG. 9 is a flowchart of processing in the playback block 204 of the moving image recording / playback apparatus according to the embodiment of the present invention.

The main reproduction block 204 is executed when highlight reproduction is instructed by the user according to an instruction from the reproduction unit 245. As shown in FIG.
The ranking data input unit 241 inputs ranking data (step 901),
The playback scene determination parameter input unit 243 inputs parameters for determining a playback scene in highlight playback (step 902). Note that this parameter includes, for example, viewing time in highlight reproduction of moving image data. Now, the playback block 204 is
Subsequently, a playback scene in highlight playback is determined by the playback scene determination unit 244 based on the parameters input by the playback scene determination parameter input unit 243 and the ranking data input by the ranking data input unit 241 (step 903). A method for determining the playback scene will be described later. Subsequently, the playback block 204 includes the playback video data input unit 24.
2, the moving image data of each playback scene is acquired (step 904), and the playback images of these scenes are displayed on the display device 103 by the display unit 246, and the playback audio is output from the audio output device 104 by the audio output unit 247. Play each playback scene by outputting (
Step 905). A method for reproducing each scene will be described later.

  Next, a playback scene determination method in the playback scene determination unit 244 will be described.

In order to determine a playback scene, the playback scene determination unit 244 first obtains the currently played position in the moving image data. This can be realized by acquiring the time of a frame in the currently reproduced moving image data when the moving image data is currently being reproduced. Alternatively, when moving image data is to be reproduced from now on, it can be realized by acquiring the time of the first frame in the moving image data. Note that the frame time in the moving image data can be realized by acquiring a time stamp corresponding to the corresponding frame if the moving image data is an MPEG stream, for example. Next, the playback scene determination unit 244 determines a playback scene from the parameters acquired by the playback scene determination parameter input unit 243, the ranking data generated by the ranking generation unit 233, and the current playback position acquired previously. Specifically, for example, when the parameter acquired by the playback scene determination parameter input unit 243 is the viewing time, first, as shown in FIG. 10A, for the ranking data generated by the ranking generation unit 233, The scene whose end position is smaller than the current playback position is deleted. In FIG. 10A, the current reproduction position is 50 with respect to the ranking data illustrated in FIG.
An example in the case of seconds was shown. Next, as shown in FIG. 10B, the times of the respective scenes are accumulated in the order of the higher rank scenes, and the playback scenes are listed within a range not exceeding the viewing time acquired by the playback scene determination parameter input unit 243. That is, the playback scenes are selected in the order of the higher rank scenes so that the total time of each playback scene does not exceed the viewing time acquired by the playback scene determination parameter input unit 243.

FIG. 10B shows an example in which the viewing time acquired by the playback scene determination parameter input unit 243 is 150 seconds. A rank 1 scene 1004 (time 5 seconds) and a rank 2 scene are shown. Scene 1005 (time 10 seconds) and rank 4 scene 1007 (time 110 seconds)
This indicates that a total of three scenes (integrated time 125 seconds) are selected as playback scenes.
At this time, as an option, the scene 1008 of rank 5 may be selected as a playback scene by cutting the scene for the remaining 25 seconds. In this case, 25 seconds from the beginning of the scene may be selected, or 25 seconds including the center of the scene may be selected. Furthermore, when the current playback position is in the middle of the scene, the time of the scene may be corrected from the current playback position to the end position. For example, in the example of FIG.
If it is seconds, the start position of the rank 2 scene 1005 is 105 seconds and the end position is 110 seconds. Now, the playback scene determination unit 244 ends with FIG.
As shown in c), the determined playback scenes are rearranged in the order of playback time, and the rearranged playback scenes are determined as final playback scenes.

  Next, the playback method of the playback scene will be described.

FIG. 11 is a flowchart for explaining a reproduction method of a reproduction scene according to the embodiment of the present invention.

As shown in FIG. 11, when playing a playback scene, first, the current playback position in the moving image data is acquired (step 1101), and the start position and end of the next playback scene are based on the current playback position. The position is acquired (step 1102). This is the playback scene determination unit 2
Of the playback scenes determined in 44, this can be realized by acquiring the start and end positions of the playback scene that is later than the current playback position and closest to the current playback position. Next, jump to the start position of the next playback scene acquired in step 1102 (step 1103).
The reproduction scene is reproduced (step 1104). This is performed by displaying the playback image in the playback scene on the display device 103 via the display unit 246 and outputting the playback audio in the playback scene to the audio output device 104 via the audio output unit 247. Is done. It should be noted that during playback of the playback scene, it is determined whether or not playback has ended periodically (step 1105). If playback has ended, playback of moving image data ends. Here, when all the playback scenes determined by the playback scene determination unit 244 have been played back, or when the end of playback is instructed by the user, it is determined that playback has ended. Otherwise, playback of the playback scene is continued. . Further, the current playback position is periodically acquired during playback of the playback scene (step 110).
6) It is determined whether or not the end position of the reproduction scene has been reached (step 1107). This can be determined by comparing the end position of the playback scene acquired in step 1102 with the current playback position acquired in step 1106. If it is determined in step 1107 that the playback scene end position has not been reached, step 11 is performed.
Steps 04 through 1107 are repeated to continue playback of the playback scene. On the other hand, as a result of the determination in step 1107, when it is determined that the end position of the reproduction scene has been reached, the reproduction scene determination unit 2 is repeated by repeating steps 1101 to 1107.
The playback scenes determined at 44 are sequentially played back, and when all playback scenes determined by the playback scene determination unit 244 have been played back, they are recognized at step 1105 and the playback ends. Thereby, only the playback scene determined by the playback scene determination unit 244 can be sequentially played back while jumping to each playback scene.

Finally, the overall flow from recording to playback in the moving image recording / playback apparatus will be described.

FIG. 12 is an example of a flowchart for explaining the flow from recording to playback in the moving image recording / playback apparatus according to the embodiment of the present invention.

As shown in FIG. 12, when recording is instructed by the user, the moving image recording / playback apparatus acquires recording of the input moving image data and its category by the recording block 201 (step 1201).
), Feature data of the input moving image data is generated by the feature generation block 202 (step 1202). Then, in the ranking generation block 203, the recording block 201
The category of the input moving image data, that is, the recorded moving image data acquired in step S3 is acquired, and the ranking of scenes in the moving image data is performed according to the category (step 1203). In addition,
Steps 1201 to 1203 may be sequentially performed after each of the steps is completed, or may be sequentially performed for each frame in the moving image data, for example. When highlight playback is designated by the user, the playback block 204 determines a playback scene based on the ranking data created in step 1203 and plays back only this playback scene, thereby performing highlight playback. Realize (step 1204).

In the moving picture recording / reproducing apparatus according to the first embodiment described above, the electronic program guide data is acquired to determine the category, and the reproduction scene can be determined according to the category. Highlight reproduction can be performed. Alternatively, it is possible to perform optimum highlight reproduction based on the category designated by the user.

  FIG. 13 is a functional block diagram of the moving image recording / playback apparatus according to the second embodiment of the present invention.

As illustrated in FIG. 13, the moving image recording / playback apparatus according to the second embodiment has a configuration including a category determination unit 218 instead of the category acquisition unit 214 in the moving image recording / playback apparatus according to the first embodiment.

The category determination unit 218 includes a voice recognition engine (not shown), recognizes voice in the input moving image data, enumerates the keywords that have appeared in advance and counts them, and the keywords appear. The category is determined according to the number of times. For example, if strikes, shoots, etc. are registered as keywords, and the keyword “strike” is recognized and counted more, the category is determined to be baseball. Similarly, if more keywords “shoot” are recognized and counted, Judge the category as soccer. And when the registered keyword does not appear, it can be set as another category. Further, the category determination unit 218 may include a caption acquisition unit (not shown). In this case, the subtitle acquisition unit acquires subtitle information sent together with the moving image data, and the category determination unit 218 uses the keyword that has appeared among the keywords registered in advance as in the case of using the voice recognition engine. Are counted and counted, and the category is determined according to the number of times the keyword appears.
Furthermore, the category determination unit 218 may include a caption image recognition engine (not shown). In this case, the caption image recognition engine acquires the caption image overlaid on the image in the moving image data and recognizes the characters, and the category determination unit 218 is registered in advance as in the case of using the voice recognition engine. Among the existing keywords, the appearing keywords are listed and counted, and the category is determined according to the number of times the keywords appear. A known technique can be applied to the voice recognition engine, the caption acquisition unit, and the caption image recognition engine.

FIG. 14 is a flowchart of processing in the recording block 201 of the moving image recording / playback apparatus according to the second embodiment of the invention.

As shown in FIG. 14, the processing flow in the recording block 201 of the moving image recording / playback apparatus according to the second embodiment is different from the processing flow in the recording block 201 of the moving picture recording / playback apparatus according to the first embodiment. However, keywords are counted in the category determination unit 218 based on the moving image data (step 603 ′). When all the moving image data is acquired and the keyword count is completed, the category is determined according to the number of times the keyword appears (step 601 ′), and the moving image data is recorded (step 604) and determined. A category table is generated based on the category (step 602).

Thereby, in the moving image recording / reproducing apparatus according to the second embodiment, the category can be determined without acquiring the data of the electronic program guide, and the optimum highlight reproduction can be performed according to the category of the moving image data. It becomes.

  FIG. 15 is a functional block diagram of the moving image recording / playback apparatus according to the third embodiment of the present invention.

As shown in FIG. 15, the moving picture recording / reproducing apparatus according to the third embodiment has a configuration in which the category determining unit 217 and the category determining unit 218 described in the second embodiment are added to the moving picture recording / reproducing apparatus of the first embodiment. .

The category determining unit 217 determines the category by the category determining unit 218 when the category acquiring unit 214 cannot acquire the category of the input moving image data, and conversely, the category acquiring unit 214 determines the category of the input moving image data. If it can be obtained, the operation of the recording unit 212 is controlled so that the category determination unit 218 does not determine the category. The flow of processing in the recording block 201 of the moving image recording / playback apparatus at this time is shown in FIG.
It becomes as shown in.

FIG. 16 is a flowchart of processing in the recording block 201 of the moving image recording / playback apparatus according to the third embodiment of the present invention.

As shown in FIG. 16, in the recording block 201 of the moving image recording / playback apparatus according to the third embodiment,
After the category of the input video data is acquired by the category acquisition unit 214 (step 601), the category determination unit 217 determines whether the category of the input video data has been acquired, thereby determining whether the category should be generated. Judgment is made (step 1601). In this step 1601, when the category determining unit 217 can acquire the category of the input moving image data, it is determined that it is not necessary to generate the category, and the moving image data acquisition (
Step 603) and recording (step 604) are performed to create a category table (step 602). On the other hand, if the category determination unit 217 cannot acquire the category of the input moving image data in step 1601, it is determined that the category needs to be generated, and the moving image data is acquired while acquiring the moving image data. The category determination unit 218
In step 603 ', keywords are counted. When all the moving image data is acquired and the keyword count is completed, the category is determined according to the number of times the keyword appears (step 601 ′), and the moving image data is recorded (step 604) and determined. A category table is generated based on the category (step 602).

Thereby, in the moving image recording / reproducing apparatus according to the third embodiment, even when the category of moving image data to be recorded cannot be acquired from the data of the electronic program guide, the category can be determined, and the optimum is determined according to the moving image data category. Highlight reproduction can be performed.

The present invention can be applied to, for example, a hard disk recorder, a video tape recorder, a personal computer, or the like that can record and reproduce moving image data.

It is an example of the hardware constitutions of the moving image recording / reproducing apparatus which concerns on the Example of this invention. 1 is a functional block diagram of a moving image recording / playback apparatus according to Embodiment 1 of the present invention. It is an example of the data structure of the electronic program guide acquired in the category acquisition part of the moving image recording / reproducing apparatus based on the Example of this invention. It is an example of the data structure of the category table produced | generated by the category table production | generation part of the moving image recording / reproducing apparatus based on the Example of this invention. It is an example of the data structure of the ranking data produced | generated in the ranking production | generation part of the moving image recording / reproducing apparatus based on the Example of this invention. It is a flowchart of the process in the video recording block 201 of the moving image recording / reproducing apparatus based on Example 1 of this invention. It is a flowchart of the process in the feature production | generation block 202 of the moving image recording / reproducing apparatus based on the Example of this invention. It is a flowchart of the process in the ranking production | generation block 203 of the moving image recording / reproducing apparatus based on the Example of this invention. It is a flowchart of the process in the reproduction | regeneration block 204 of the moving image recording / reproducing apparatus based on the Example of this invention. It is a figure explaining the transition of the determination method and data processing of the reproduction | regeneration scene based on the Example of this invention. It is a flowchart explaining the reproduction | regeneration method of the reproduction | regeneration scene based on the Example of this invention. It is an example of the flowchart explaining the flow from recording to reproduction | regeneration in this moving image recording / reproducing apparatus which concerns on the Example of this invention. It is a functional block diagram of the moving image recording / reproducing apparatus which concerns on Example 2 of this invention. It is a flowchart of the process in the video recording block 201 of the moving image recording / reproducing apparatus which concerns on Example 2 of this invention. It is a functional block diagram of the moving image recording / reproducing apparatus which concerns on Example 3 of this invention. It is a flowchart of the process in the video recording block 201 of the moving image recording / reproducing apparatus which concerns on Example 3 of this invention.

Explanation of symbols

100 ... Broadcast data input device 101 ... Central processing unit 102 ... Input device
103 ... Display device, 104 ... Audio output device, 105 ... Storage device, 106 ...
Secondary storage device 107 ... bus 108 data transmission / reception device 201 recording block 202 feature generation block 203 ranking generation block 20
4... Playback block, 211... Recorded video data input unit, 212.
3 ... Recording unit, 214 ... Category acquisition unit, 215 ... Category table generation unit,
216 ... Category data holding unit, 221 ... Analyzed video data input unit, 222 ...
Feature generation unit, 223 ... feature data holding unit, 231 ... feature data input unit, 232
..Category data input unit, 233 ... Ranking generation unit, 234 ... Ranking data holding unit, 235 ... Baseball ranking generation unit, 236 ... Soccer ranking generation unit, 237 ... Drama Ranking generation unit, 241... Ranking data input unit, 242... Playback video data input unit, 243.
44... Playback scene determination section, 245... Playback section, 246... Display section, 247.
Audio output section

Claims (9)

In a moving image recording / reproducing apparatus including a storage unit that stores moving image data received as broadcast data or acquired via a network, and a reproducing unit that reproduces moving image data.
A feature data holding unit for holding video or audio features for each frame constituting the moving image data stored in the storage unit;
Category acquisition means for acquiring a category of moving image data stored in the storage means;
A ranking generator according to category,
A ranking data holding unit that holds a scene including one or more frames that are candidates for highlight reproduction and its ranking;
In response to a highlight reproduction instruction including designation of a reproduction condition from the outside, a reproduction scene is determined based on the designation of the reproduction condition and the ranking as to whether or not each scene held in the ranking data holding unit should be reproduced. A playback scene determination means,
The scenes and rankings held in the ranking data holding unit are classified into feature data for each frame held in the feature data holding unit by the ranking generation unit activated according to the category acquired by the category acquisition unit. Based on an algorithm according to the category, a scene including one or more frames is extracted, and the extracted scene is ranked and generated.
The moving picture recording / reproducing apparatus characterized in that the reproduction means reproduces two or more reproduction scenes determined by the reproduction scene determination means in order of reproduction time. In claim 1,
When the highlight reproduction instruction is received during the reproduction of the moving image data by the reproducing unit, the reproduction scene determining unit obtains the reproduction position of the moving image data reproduced by the reproducing unit, and A moving image recording / reproducing apparatus, wherein a reproduction scene is determined based on a reproduction position, designation of the reproduction condition, and the ranking. In claim 2,
The reproduction scene determination unit extracts a scene having an end position later than the reproduction position of the moving image data reproduced by the reproduction unit from the scenes held in the ranking data holding unit, and determines a reproduction scene. A moving image recording / reproducing apparatus. In claim 1,
Including the specification of viewing time of the playback scene as the playback condition,
The moving image recording / playback apparatus, wherein the playback scene determination means selects a scene with a high ranking so that the total playback scene time does not exceed the instructed viewing time. In claim 1,
When the category acquisition unit acquires the category of the moving image data, the feature data is read from the feature data holding unit, and one or more frames that are candidates for highlight reproduction of the moving image data are selected according to the acquired category. A moving picture recording / reproducing apparatus characterized in that the scenes included and the ranking thereof are extracted and the results are held in the ranking data holding unit. In claim 1 or 5,
The moving image recording / reproducing apparatus characterized in that the category acquisition means acquires an electronic program guide and acquires a category of the moving image data from the electronic program guide. In claim 1 or 5,
Furthermore, it has a voice recognition means for recognizing the voice included in the video data,
The category acquisition means recognizes the voice included in the moving picture data by the voice recognition means, counts the number of appearances of a predetermined keyword, and determines the category of the moving picture data based on the count result. A moving image recording / reproducing apparatus. In claim 1 or 5,
Furthermore, it has subtitle data acquisition means for acquiring subtitles included in the video data,
The category acquisition unit acquires a subtitle character string included in the video data by the subtitle acquisition unit, counts the number of occurrences of a predetermined keyword, and determines the category of the video data based on the count result A moving image recording / reproducing apparatus. In claim 1 or 5,
The moving picture recording / reproducing apparatus, wherein the category is designated by a user of the moving picture recording / reproducing apparatus.

Images