JP3341574B2 - Video signal recording / reproducing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for confirming the contents in a short time from a large amount of various image information mainly represented by a television broadcast, a digital television broadcast, and a video server. The present invention relates to a video signal recording / reproducing device capable of performing the following.

[0002]

2. Description of the Related Art Today, information collectively referred to as television programs and multimedia is flooding, and this tendency tends to increase day by day. Programs that the viewer generally considers to be of low value are deleted, and there is a human original desire to concentrate on programs that are useful, entertaining and enjoyable, but sometimes those programs are missed There is. Even in the case of image information collectively referred to as package media, there is a reality that a large amount of information overflowing due to lack of time cannot be sufficiently accessed and utilized. As means for avoiding these, there are means such as a program guide, a newspaper, and a preview. There is also a means of recording on a video tape and using it at another free time.

Further, a high-speed reproduction function for saving time and viewing is known. On the recording side, there is a digest version obtained by subjecting the original source to some kind of secondary processing, and such a digest version is often used in television sports programs and the like. However, this digest version takes a long time to edit, and ordinary household users do not have such facilities, and their use at home is limited.

As one of means for solving these problems, there is a means usually called a ring buffer. For example, a continuous image can be recorded for 60 seconds, and the image can be viewed retrospectively when necessary. Examples of the ring buffer include a hard disk and a semiconductor memory (flash memory, IC memory, DRAM, ferroelectric memory) and the like. There is a system operated as a recording / reproducing medium.

Although this system is a means for interpolating an oversight, the recording / reproducing time is limited by the capacity of a memory used, and it is not always possible to secure a necessary and sufficient recording / reproducing time.

At present, there is a tape streamer which can be used in the market and is viewed from the viewpoint of obtaining more recording capacity. This tape streamer is based on a magnetic recording tape, 2.5 gigabytes, and based on optical recording (a type in which an optical recording medium is applied to a tape-shaped base film), and is a terabyte. Capacity has been realized. By using such a means, it is possible to record for a longer time. However, in the case of the search, access is time-consuming and complicated even though there is a means such as using a search data code. For this reason, except in special cases, it is not generally used and is exclusively used for computer data backup.

Further, in a system using a hard disk, a system has been constructed in which a large capacity is secured and the easy accessibility of the disk is emphasized. In order to realize such a large capacity, there is a system in which a plurality of small-capacity disks are configured in a stack and used for a video server. This system, RAID (Red u ndant Array ofIn
Currently used as expensive disks).

Further, in order to secure desired information,
It is necessary to reduce the redundancy of the image, and for this purpose, using image processing technology such as motion vector detection, to detect scene changes in the image and use it for cut-based editing or browsing (display) There is.

As a well-known method of detecting the motion vector, there is a block matching method first. In this block matching method, as shown in FIG. 7, a reference frame 51 and a search frame 53 are used to subtract all pixels of an image (m × n) of an inspection block 55 and a reference block 52 in a search area 54. Do
An absolute value is obtained, a product-sum operation is executed, the minimum value of the residual is obtained within a search range, and the relative position is used as a motion vector.

Second, there is an integral oblique method. In the integral oblique method, two-dimensional data is obliquely projected one-dimensionally to simplify an arithmetic circuit. Take the total addition of horizontal pixels and vertical pixels of all pixels of the inspection block size,
As described above, the minimum value of the residual is obtained and used as a motion vector.

[0011]

Although it is not possible to access all information at once for information transmitted continuously, as in television broadcasting and digital television broadcasting, all information is stored once. If there is a recording / reproducing medium to use, the utilization of such information can be expanded. From this point of view, if means for selectively recording desired information in the process of recording a video signal and deleting and not recording redundant parts is installed in a recording / reproducing device, it will be possible to reduce the time required in a flooded information society. Can access and use more information at

Conventionally, there has been a so-called time-saving video in which the time for viewing is shortened, but this video records all originally too much information. Therefore, the function of selectively and generally accessing necessary information in a short time has been limited. For these reasons, it is hard to say that this time-saving video is suitable for the original way of utilizing information. In view of the above, the present invention is to realize an information collecting means which has an information selection function at the time of recording and a thinning function of a video signal thereby, and provides an overview of the entire information in a short time.

Further, it is difficult to realize the scene change detection by reducing the redundancy of the image and securing the desired appropriate information by focusing only on the image processing technology. Then, an algorithm for securing necessary and sufficient functions becomes complicated and the device becomes enormous. For this reason, there is no commercial operation at this time. Therefore, the present invention intends to realize a device that can be put to practical use by interpolating insufficient elements using image processing technology and speech recognition technology as redundancy reduction means by thinning out spatial images in the time axis direction. Things.

The algorithm used to detect or extract the scene of the image used at this time is a system that includes flexibility and flexibility that takes into account the convenience of viewing or utilizing recorded information later. It is necessary to have user option function. In other words, in a video of a certain genre such as video on demand,
In particular, areas of high user interest should be focused on, even if they become redundant. Therefore, the present invention realizes a function in which an algorithm for feature extraction can be adaptively changed in conjunction with a scene change extraction keyword that can be requested in advance.

[0015] Accordingly, an object of the present invention is to provide an image processing apparatus which is based on thinning-out of an image at regular intervals and which is considered to be important when a general overview of the contents of a large amount of various image information is made in a short time. Certain scenes are characterized by thinning out at variable intervals. In addition, by selecting less mediocre images from redundant information and extracting highlight scenes in a focused manner, and by recording / reproducing them, a short time is saved. It is an object of the present invention to provide a video signal recording / reproducing apparatus capable of overviewing the entire information.

[0016]

SUMMARY OF THE INVENTION The first aspect of the present invention, in the video signal recording / reproducing apparatus that records on a recording medium body by extracting a portion of the video signal temporally spatially graphical interface Display on the display device by
First and second extracting the video signal in the menu
Means for inputting extraction processing information, and first extraction processing information
Means to extract video signal at fixed time intervals set in
And that the second extraction processing information is detected in the video signal
Video signal at a time interval that is denser than a certain time interval
Means for extracting, extracted Film image signal only recording medium
And a means for recording to a video signal.

A part of an image is automatically thinned out at a certain time interval (interval) from an input video signal. For example,
Subtract a part of the field / frame from the binary arrayed intervals. On the other hand, there is a highlight scene in a sports competition that competes for acting within a certain time, and there is a highlight scene that can be predicted in advance. Intervals are combined to increase the recording density in a specific area due to such elapsed time. Further, in a highlight scene that cannot be predicted, since there is a peculiar spectrum or volume represented by the sound of the pistol at the start, the voice recognition means is also used and the thinning out of the interval is made variable. Further, a specific scene in the input image is extracted by a keyword selected by the user on the menu screen, and the thinning time of the image in the extracted specific scene is made variable. These specific restricted appropriate scenes are recorded on the recording / playback medium. Incidentally, Japanese Patent Application Laid-Open No. 03-807
No. 82 discloses a method of extracting features of an audio signal, comparing the features with a previously stored feature, and extracting a corresponding video signal based on the result. No method is disclosed for using the extraction sequence and the motion of the image.

[0018]

An embodiment of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows an embodiment of a video signal recording / reproducing apparatus according to the present invention. A signal selected from a plurality of television broadcasts is input. The input signal is a composite video signal in which the audio signal and the video signal are composited. The input composite video signal is supplied to a voice recognition circuit 1, a motion vector detection circuit 2, a synchronization signal separation circuit 3, and a gate circuit 5. In the voice recognition circuit 1,
As will be described later, a singular point is extracted and its volume is also analyzed. The analysis result is supplied to the microprocessor 9. The motion vector detection circuit 2 is controlled by the microprocessor 9, and detects a motion vector of the video signal separated from the input composite video signal as described later. A feature point is extracted from the detected motion vector, and an interrupt signal d1 is generated according to the feature point by a motion vector detection circuit 2
Is supplied to the gate signal generation circuit 6 from

In the synchronizing signal separating circuit 3, the synchronizing signal is separated from the supplied input video signal, and the separated synchronizing signal is supplied to the synchronizing signal generating circuit 4. The synchronization signal generation circuit 4 generates a synchronization signal in response to the supplied synchronization signal. The generated synchronization signal is supplied to a synchronization signal generation circuit 4
To the time lapse command signal generation circuit 7. The time lapse command signal generation circuit 7 is controlled by the microprocessor 9 and supplies a command signal d2 having passed a predetermined time from the supplied synchronization signal to the gate signal generation circuit 6.

The gate signal generation circuit 6 generates a gate signal based on the interrupt signal d1 and the command signal d2. For example, in the gate signal generation circuit 6, a gate signal is generated by the logical sum of the interrupt signal d1 and the command signal d2. The generated gate signal is supplied to the gate circuit 5. In the gate circuit 5, the input composite video signal is gated in units of an integral multiple of the field or frame of the input composite video signal in accordance with the supplied gate signal. The output signal from the gate circuit 5 is supplied to the time-lapse recording / reproducing medium 8 as a discrete video signal. That is, the gate circuit 5 performs sampling of the video signal at a controlled interval, and thereby thins out in the time direction. The time lapse recording / reproducing medium 8 is controlled by the microprocessor 9 to record / record the supplied input video signal.
Playback is performed.

As described above, the microprocessor 9 controls the motion vector detection circuit 2 and the time lapse command signal generation circuit 7, and the feature points extracted from the video signal and the singular points of the audio signal are always checked. Specifically, if there is a desired feature point or singular point, its output (interrupt signal d1 and command signal d2) is output to OR gate circuit 6.
And any of its signals d1 and / or d2
Thus, the output of the gate circuit 5 is gated in a unit of an integral multiple of a field or a frame. The gated output becomes a discrete video signal, which is supplied to the time-lapse recording / reproducing medium 8 and recorded. The video signal reproduced from the time-lapse recording / reproducing medium 8 is displayed on a television monitor 10. A menu screen 11 is also displayed on the television monitor 10 as described later.

Next, FIG. 2 shows a detailed block diagram of the motion vector detecting circuit 2 according to the present invention. The motion vector detecting circuit 2 includes a horizontal / vertical pixel thinning circuit 21, a horizontal / vertical spatial filter 22, a projection circuit 23,
It comprises a correlation detection operation circuit 24, a control / discrimination circuit 25, a field memory 26 and a line memory 27.
First, the video signal separated from the input composite video signal is supplied to the horizontal / vertical pixel thinning circuit 21. In the horizontal / vertical pixel thinning circuit 21, the pixels of this video signal are thinned, for example, by half both horizontally and vertically. Pixels have been decimated in half horizontally and vertically, that is, the number of pixels is 1 /
The video signal 4 is subjected to an image processing technique in a horizontal / vertical spatial filter 22, that is, to a two-dimensional spatial filter, and is supplied to a projection circuit 23.

The projection circuit 23 uses the data in the field memory 26 and the line memory 27 to detect a search area for detecting a motion vector in an image. The line memory 27 is, for example, a memory for 1H. The correlation detection operation circuit 24 uses, for example, correlation detection of two types of images, as described later. This calculation result is supplied from the correlation detection calculation circuit 24 to the control / discrimination circuit 25. The control / discrimination circuit 25 controls the motion vector detection circuit 2 according to an instruction from the microprocessor 9. further,
The control / discrimination circuit 25 determines whether or not the supplied operation result is a motion vector. If it is determined that the result is a motion vector, an interrupt signal d1 is supplied to the gate signal generation circuit 6.

FIG. 3 shows a block diagram of the speech recognition circuit 1 of the present invention. The input composite video signal is supplied to the audio recognition circuit 1. The speech recognition circuit 1 includes a speech separation circuit 31 and a DSP (FFT) spectrum /
It is composed of a volume analysis circuit 32. The input composite video signal supplied to the voice recognition circuit 1 is supplied to a voice separation circuit 31. The audio separation circuit 31 separates an audio signal from the supplied input composite video signal, and supplies the separated audio signal to a spectrum / volume analysis circuit 32. In the spectrum / volume analysis circuit 32, a DSP (Digital
FFT (Fast Fourier T) using Signal Processing
ransformatio n ) conversion and a distinctive representative spectrum is detected from the supplied audio signal. A singular point is extracted from the detected spectrum, and its volume is also analyzed. The analysis result is supplied from the speech recognition circuit 1 to the microprocessor 9. The microprocessor 9 controls the time lapse command signal generation circuit 7 based on the analysis result. As an example, a thinning time interval of an image and a time interval (interval) between fields / frames are controlled.

Here, an example of the menu screen 11 is shown in FIG. First, in the genre 41, as shown in the figure, each genre of sports is displayed. For example, when golf is selected from this sport genre, keywords related to golf are displayed as keywords 42 as shown in the figure. Then, a desired keyword is selected from the keywords 42, and an image related to the selected keyword is displayed on the television monitor 10. At this time, a plurality of keywords can be selected. FIG. 5A shows a standard time series of an input video signal as an example of the displayed video signal.

Then, a desired interval is set in consideration of the keyword selected using the time lapse time 43. FIG. 5B shows an example of a time series of a discrete video signal in which an interval is set by the time lapse time 43. In this time lapse time 43,
The intervals are taken into consideration and displayed according to the genre of the video signal, for example, the type of sports. The user selects a substantially optimal initial value of the interval from the displayed intervals in a menu format.

The microprocessor 9 determines the analysis result from the speech recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2 shown in FIG. 3, and extracts a feature point. The result is converted to a desired value. Upon arrival, the time series of the discrete video signal shown in FIG. 5B changes to a coarse and dense time series as shown in FIG. 5C.

Further, a prearranged time interval different from a certain interval is included in the time keeper 44 in the time zone selection 45 and the frame number 46.
Is set using. The prearranged time interval is a combination of intervals that increases the recording density in a specific area based on the set interval. A time zone is set at a predetermined time using the time zone selection 45, and the required number of fields / frames can be set using the number of frames 46 for each of the plurality of set time zones. FIG. 5D shows an example of the resulting discrete video signal. FIG. 5D shows three time zones tα1 to tα2, tα3 to tα4, and tα5 to tα.
6 are set, and the time of the set time zone is also
The interval and the required number of frames are set independently.

Further, tβ1 and tβ2 shown in FIG.
The microprocessor 9 determines the analysis result from the speech recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2 shown in FIG. 3 and extracts a feature point. Is an example in which a characteristic scene, that is, a field / frame, is added to a video signal set using a prearranged time interval when the video signal reaches a predetermined time.

As described above, the video signal input as shown in FIG.
It becomes a discrete video signal as shown in FIG. 5B. Then, a feature point is extracted according to the analysis result from the voice recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2, and when the extracted feature point reaches a desired value, the feature point is extracted. The series changes to a coarse and dense time series as shown in FIG. 5C or FIG. 5D. Therefore, a specified temporal region from the input video signal is recorded in a time-lapse manner, that is, as shown in FIGS. 5B, 5C and 5D, or at a period of a standard video signal in a relatively short time. / Recorded on a reproduction medium 8, for example, a hard disk, a time lapse VTR, or the like.

More specifically, the time lapse time 4
Although the video signal is controlled to be recorded / reproduced at the interval of the initial value set in step 3, the interrupt function operates according to the interrupt signal d1 from the motion vector detection circuit 2. If the extracted feature point reaches a desired value even during operation at an interval (time-lapse mode) set as an initial value by this interrupt function, a scene with the feature is added (variable timing lapse). Mode) automatically switches to operation. And
When the interrupt signal d1 disappears, the mode automatically returns to the time lapse mode of the original set value.

Here, the above-described motion vector detection circuit 2
The algorithm of the motion vector detection used in the above will be briefly described. In this motion vector detection, correlation between two types of images is used together. Ordinary unnormalized correlation,
Two of the normalized correlation functions. The former is a product-sum operation of N pixels, which is the same type as convolution, as shown in equation (1).

[0033]

(Equation 1)

This equation (1) has a high arithmetic processing speed, but as a drawback, for example, when the image is uniformly white, the maximum value is detected even if the image is not similar at all. On the other hand, the normalized correlation function expressed by the equation (2) has an advantage that even if there is a linear transformation (a constant gain or offset), the gain or offset is not affected.

[0035]

(Equation 2)

However, equation (2) has a disadvantage that it takes a long time to perform the arithmetic processing. Therefore, it is controlled by the microprocessor 9 and used properly in some cases. The microprocessor 9 controls the motion vector detection circuit 2 by an algorithm that takes into account the average value and the minimum value of the correlation operation values and their temporal changes. Here, N used in the equations (1) and (2) is N pixels, R is an original pixel serving as a reference, and M is a pixel in an area where motion is to be detected.

From the viewpoint of improving the processing time of the correlation operation value of the motion vector detection circuit 2, as an example, a search area specialized for each keyword of the sports genre is set. For this purpose, as a pre-process, the image is obtained by thinning out pixels, applying a spatial filter, removing high-frequency components in the background, and adding dilation processing (geometric manipulation, processing to enlarge the area around the object) if necessary. Feret diameter at
Find X and Y. The Feret diameter X is a horizontal diameter obtained by projecting the image on the coordinate axis X, and the Feret diameter Y is a vertical diameter projected on the coordinate axis Y. Here, the value of the Feret diameter is made to match the search area.

Therefore, as shown in FIG. 6, the lengths of the horizontal coordinates X1, X2 and the vertical coordinates Y1, Y2 projected on the coordinate axes X and Y can be known. The coordinates are matched with the detection area and search area of the pattern matching method used for motion vector detection.

The correlation detection method used in the motion vector detection is extended to a pattern matching function in this embodiment. For example, the menu screen 1 shown in FIG.
In 1, the present invention is applied to a case where a specific scene such as a golf putter scene from a sports genre and a scene similar thereto are recorded on the time-lapse recording / reproducing medium 8.

As described above, in this embodiment, the algorithm is constructed such that similar highlight scenes related to a specific keyword are substantially extracted. In this case, the geometrical form in the image of the putter operation, for example, a few representative examples from the address to the follow-through are registered as a reference template, compared with the input video signal, and similar. The scene is controlled so as to extract all even if it is somewhat redundant.

The camera angle in the case of sports is subject to physical restrictions according to the genre. For example, up and down in ski competition, diagonal in marathon,
As in the side, many phenomena and vectors can be foreseen. Therefore, a function is provided to allow the user to specify the detection area while looking at the lattice-like blocks appearing in the menu.

In particular, in a sports competition or the like, a highlight scene exists, and it is not only information that merely includes a change in movement. The audience, or the information you want to use, is up to you. In other words, the way in which information is used, used, and enjoyed differs individually. Therefore, these long-running programs are digested in a short time,
When an overview is desired, the necessary decisive scene needs to have a function of automatically selecting an outline image based on an individual's selection. Therefore, the area of motion area, direction (vector), speed of motion,
Furthermore, a template and an extraction algorithm thereof are installed in the system in advance in consideration of attributes such as information on color. Therefore, an appropriate extraction function is realized, and the practicality is enhanced.

As an example of extracting the highlight scene, in golf, a sharp sound resounding at the moment of hitting a golf ball with a driver, in baseball, a sound of hitting a ball with a bat, and in soccer, when a ball is kicked with a corner kick. There is a sharp pop, or a high pitched or loud voice when the announcer shouts at goal, home run, intercept, etc. With such information of voice and impulsive sound as a medium,
It is characterized in that it is added to the video signal as additional information for accurate feature point / singular point extraction, and is further weighted with a genre, a keyword, and the like, and added to a highlight signal or a phenomenon that occurs in conjunction with movement.

In this embodiment, the motion vector detecting circuit 2
The gate signal is generated by using the interrupt signal d1 from the CPU and the command signal d2 from the time lapse command signal generation circuit 7. The motion vector detection circuit 2 and the time lapse command signal generation circuit 7 are controlled by the microprocessor 9 Therefore, the command signal d2 of the time lapse command signal generation circuit 7 can be directly supplied to the gate circuit 5 to generate a discrete video signal.

[0045]

According to the present invention, it is possible to avoid the inconvenience of viewing keywords of a certain genre over a long period of time,
A wide range of information can be accessed in a short time, and the range of information utilization can be drastically expanded.
Further, it is possible to selectively extract substantially desired information selected by the user from the input video signal, improve the quality of usable information, and record / reproduce them.

[Brief description of the drawings]

FIG. 1 shows an embodiment of a video signal recording / reproducing apparatus to which the present invention is applied.

FIG. 2 is an example of a motion vector detection circuit to which the present invention is applied;

FIG. 3 is an example of a speech recognition circuit to which the present invention is applied;

FIG. 4 is an example of a menu screen to which the present invention is applied.

FIG. 5 is a schematic diagram showing a time series of a video signal used for describing the present invention.

FIG. 6 is a schematic diagram used to explain motion vector detection to which the present invention is applied;

FIG. 7 is a schematic diagram used for explaining a block matching method.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input video signal selection circuit, 2 ... Motion vector detection circuit, 3 ... Synchronization signal separation circuit, 4 ... Synchronization signal generation circuit, 5 ... Gate circuit, 6 ... OR gate , 7 ... time lapse command signal generation circuit, 8 ...
Time-lapse recording / reproduction medium, 9: microprocessor, 10: television monitor, 11: menu screen

────────────────────────────────────────────────── ─── Continued on front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/76-5/956

Claims (2)

(57) [Claims] 1. A video signal recording / reproducing apparatus that records on a recording medium body by extracting a portion of the video signal temporally spatially, is displayed on the display device by the graphic interface
First and second extractions for extracting video signals
Means for inputting the output processing information, at the fixed time interval set by the first extraction processing information
Means for extracting the video signal, wherein the second extraction processing information is detected in the video signal.
At a time interval that is denser than the fixed time interval
Means for extracting the video signal, means for recording only movies image signals of the extracted to the recording medium
And a video signal recording / reproducing apparatus. 2. The video signal recording / reproducing apparatus according to claim 1 , further comprising: means for extracting a characteristic change such as audio information; motion detection information in the video signal; Time interval for extracting the video signal according to the temporal change
There video signal recording / reproducing apparatus according to claim Rukoto varied so that dense.