KR100865683B1 - How to store multidimensional scalable video data - Google Patents

Abstract

The present invention relates to a method of storing multi-dimensional scalable video data, and more particularly, to a method of storing video data on a disk on which a search operation is performed to transmit multi-dimensional scalable video data to a streaming service. More specifically, when partially extracting a frame according to the level of scalability, it is possible to prevent a search delay caused by a plurality of search operations, thereby reducing the overhead of the disk, thereby reducing the overhead of the server Multi-dimensional scalable video data, which increases throughput of memory and disk, reduces the time for reading excess data that is not required for decoding, and thus enables the server to provide a smooth streaming service upon client request. To provide a storage method of the technology Configuring a first step for separating the unit streams in accordance with the scale level to control the resolution of the multi-dimensional scalability, the video stream; Arranging the unit streams in units of streaming service periods; And rearranging the unit stream to store video data on the disk in such a manner that the read data to be read from the disk is minimized during the streaming service cycle.

Description

Data Placement Scheme for Mulit-Dimensional Scalable Video Data

1 is a block diagram schematically illustrating a method for storing multidimensional scalable video data according to the present invention;

2 is a block diagram schematically illustrating a method of storing multidimensional scalable video data according to the present invention;

3 is a block diagram schematically illustrating a method of reducing the amount of computation of the single stream rearrangement method of FIG.

4 is a diagram illustrating a unit stream configuration of a scalable video stream having spatial scalability and temporal scalability according to the present invention.

5 is a diagram schematically illustrating a storage state of general video data and a procedure of storing video data in a method of storing multidimensional scalable video data according to the present invention.

6 is a diagram illustrating an embodiment to which a method of storing multidimensional scalable video data according to the present invention is applied.

FIG. 7 illustrates an embodiment to which a method of storing multidimensional scalable video data according to the present invention is applied; FIG.

The present invention relates to a method for storing multi-dimensional scalable video data, and more particularly, to store video data on a disk on which a search operation is performed to transmit multi-dimensional scalable video data to a streaming service. It is about how to.

In general, scalable video is video compressed to be adaptively transmitted and reproduced according to various network environments, terminal capacities, and user preferences, and provides various resolutions using only a single compressed video stream.

Here, multi-dimensional scalable video such as H.264 / MPEG-4 SVC has spatial scalability to provide video having various screen sizes, temporal scalability to provide video of various frame rates, specific frame rate and specific screen. It provides multi-dimensional scalability such as SNR scalability that provides various quality video by adjusting bit-rate while maintaining the size of.

In addition, multidimensional scalability may be simultaneously applied, and a server providing scalable video extracts and provides a scalability level corresponding to a request to a client requesting a specific resolution.

However, since scalable video partially extracts frames and the like according to the scalability level, when partial frames are extracted from successive frames, a plurality of search operations are performed during a streaming service cycle, and a search delay (where a search operation occurs) is performed. Seek Latency acts as a major cause of disk overhead, which can cause server overload, degrades memory and disk throughput, and does not require excess data for decoding. There was a problem such as increased reading time.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a method for storing multi-dimensional scalable video data which maximizes the utilization of the disk by minimizing the excess data read additionally to reduce the disk overhead. It aims to do it.

Another object of the present invention is to provide a method for storing multidimensional scalable video data which maximizes disk utilization and reduces the amount of data buffered in the memory of a video server system.

In order to achieve the above object, the present invention provides a method comprising: a first step of dividing a multidimensional scalable video stream into unit streams according to a scalability level of adjusting resolution; Arranging the unit streams in units of streaming service periods; Rearranging the unit streams and storing video data on the disk to minimize read data that should be read from the disk in a streaming service period; It includes.

Here, in the third step, the rearrangement of the unit streams may be performed to calculate read data according to all cases of the unit stream array, and may be an array having the minimum read data.

In addition, in the third step, the rearrangement of the unit streams selects an arbitrary unit stream from the unit stream array, and calculates read data according to each case while positioning the position from the start point to the end point of the unit stream array, The calculated read data may be a minimum array.

When selecting an arbitrary unit stream, one of the last unit streams is selected from the first unit stream, and all unit streams of the unit stream array are selected once.

In addition, the read data may be changed according to each streaming service cycle.

In addition, the scalability level is temporal scalability to adjust the frame rate of the video; Spatial scalability to resize the screen to play video; SNR scalability to adjust bit rate coding video data; Characterized in that comprises a.

The unit stream is a partial stream of data corresponding to a picture group (GOP), and is a minimum data block capable of generating a video having increased and decreased levels of scalability.

In addition, when a plurality of picture groups are included in the streaming service period, the unit stream in each picture group may be mixed to minimize read data.

In addition, the re-arranged unit streams in the third step are indexed in a mixed state and stored on a disk, and when a streaming service is provided, the streams may be loaded in memory in order according to the indexes.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating a method of storing multidimensional scalable video data according to the present invention, and FIG. 2 is a block diagram schematically illustrating a method of storing multidimensional scalable video data according to the present invention. As shown in the figure, the multi-dimensional scalable video data storage method according to the present invention is performed as follows.

First, a video data source is classified into a unit stream according to a scalability level desired by a user (S10).

Here, the encoded scalable video data is arranged in the order of decoding by the decoder and stored in a storage device such as a disk, which is stored in units of frames.

In addition, the data is divided and stored for each unit stream according to a scalability level desired by the user, wherein the scalability level is a spatial scalability capable of providing a video having various screen sizes and various frames. Temporal Scalability to provide frame rate video, and SNR Scalability to provide video of various quality by adjusting bit-rate while maintaining a specific frame rate and specific screen size. It provides multi-dimensional scalability such as Signal to Noise Ratio Scalability, and enables to apply such multi-dimensional scalability at the same time.

Therefore, in the step S10, when the user requests the video data of 128 frames per second and the size of the cellular phone, the spatial scalability of the scalability level is the cell phone screen size, the temporal scalability is 300 frames, and the SNR extension. The surname is 128, so that the original video data is classified into a unit stream for delivery to the user according to the multidimensional scalability level.

Video data sources classified into unit streams are classified by the amount to be sent during a single streaming service cycle (S20).

Here, the streaming service transmits data to a user at a predetermined cycle. The streaming service is a technique for reproducing voice or video animation in real time, and downloading and playing files such as video data to a hard disk drive without downloading. It is a technique that plays in real time.

Also, for streaming to work, the client receiving the video data must be able to collect the video data and send it continuously to an application that converts the data into sound and pictures. If it is too fast, the extra data is stored in the buffer and streamed at the same time. On the contrary, if the data reception speed is too slow, the data must be reproduced in real time without receiving the data, so the representation of the data is not smooth.

 At this time, the server side sends the video data to the client at regular intervals. For example, if you are streaming 30 minutes of video, the server side sends 6 times the periodicity to play for 5 minutes, and requests streaming to the server. It sends data to other clients with a certain period of time for extra time so that streaming requests from multiple users can be performed.

Thus, in step S20, the amount of data to be transmitted to the client is identified during the streaming service period, and the unit stream is classified by the amount of data to be transmitted. For example, the streaming service period is 1 minute, and the total video data to be transmitted is 100. If the video data that can be transmitted in one minute is 10, it is separated by as much as 10 video data.

In addition, when storing the classified unit stream on the disk, it stores the seek operation according to the level of scalability desired by the user (S30, S40).

Here, for example, the streaming service cycle is 1 minute, the total video data to be transmitted is 100, and the video data that can be transmitted for 1 minute is divided into 10, the video data to be transmitted in 10 units of a single service period 1 minute In order to minimize the seek operation, the amount of data to be read from the disk should be stored to a minimum, and the arrangement of the unit streams should be changed.

The reason is that the unit stream is basically stored in order and not stored according to the scalability level desired by the user. Therefore, in order to read the data according to the scalability level required by the client, the original data is read in order. Seek operations should also be performed on data that is not needed, thereby changing the array of unit streams to minimize the seek operation.

To this end, a method of storing multidimensional scalable video data according to the present invention will be described with reference to Equations 1 and 2 below.

Here, a GOP is a group of pictures, which is a set of pictures including at least one intra-coded picture in MPEG image processing, and the MPEG moving picture compression algorithm is a JPEG. It is a form that developed the compressed image compression algorithm and H.261 moving image compression algorithm, etc., and the image is a frame or field composed of a luminance signal and two chrominance signals, from an I image compressed with a JPEG purified image and an image immediately before the end of encoding. It consists of three pictures: P picture (Predictive-coded picture) that predicts the change in block unit, and B-directional picture (B picture), which is interpolation between successive front and back adjacent pictures. It is used as a unit of picture editing, and MPEG video is a combination of these pictures in a certain pattern.

In the case of video data, each screen is played by referring to the previous screen, whereas the GOP can play the screen as an independent object without having to refer to the front and back screens. It includes unit streams according to gender level.

In addition, the minimum data block that can be used to convert the GOP to any scalability level, that is, to video having any scalability level, is defined as a unit stream.

Equation 1 represents a unit stream required for constructing a single GOP according to the scalability level, and represents a multidimensional scalability level as (i, j, k), where i represents a level of spatial scalability, and j is K denotes the level of SNR scalability.

Further, GOP ^k (L, M, N) denotes a k-th GOP having a scalability level of spatial scalability L, temporal scalability M, and SNR scalability N, and u ^k (i, j , k) represents the unit streams required to construct a GOP.

In the case of using FGS (Fine Grain SNR Scalability), which provides continuous scalability by the SNR scalability method, the enhancement layer is divided into a plurality of layers according to the application to further expand in the SNR scalability direction. You can create a castle level.

Then, Equation 2 is used to calculate read data according to the level of scalability desired by the user.

Where L, M, and N are the maximum values of the scalability level of each dimension, and C (i, j, k) is the number of service requests having the scalability level of (i, j, k), and Rs (i, j, k) represents the amount of data that must be read from disk in order to provide a stream corresponding to the request of the scalability step (i, j, k) in one service period.

For example, if (3, 1, 2), the data that must be read from disk to provide a stream corresponding to a request with a spatial scalability level of 3, a temporal scalability level of 1, and an SNR scalability level of 2 To calculate the amount of.

If the spatial scalability level is 3, the unit streams according to 1 and 2 should be read. If the SNR scalability is 2, the unit stream corresponding to 1 should be read together with 2 as well.

Therefore, an array of unit streams arranged in order is arranged according to the number of all cases, and according to the scalability level, the amount of data Rs to be read is calculated, and the amount of read data Rs is minimized. Arrays of unit streams are stored on disk.

In this case, the order of the unit streams may be reversed because they are not in order, but the unit streams are indexed and stored in an array that minimizes the search operation. .

3 is a block diagram schematically illustrating a method of reducing the amount of computation of the single stream rearrangement method of FIG. 2. As shown in the figure, the method of reducing the amount of calculation of the single stream rearrangement method according to the present invention selects the unit stream having the position determined (S30-1), and arranges the unit stream at the position where the read data is the minimum. (S30-2), the above steps (S30-1, S30-2) are repeated until the positions are determined for all the unit streams, and if the positions are determined for all the unit streams, the process ends (S30-3).

Here, in the method for storing multidimensional scalable video data according to the present invention, in order to most accurately reduce a search operation, all cases of an array of unit streams are obtained, and the amount of read data is calculated and selected as the minimum array.

At this time, it can be calculated as an optimal value, but the amount of read data is not optimally minimized, but a method of increasing the speed by reducing the amount of computation is added.

Randomly select one of the unit streams included in one service cycle, and change the position of the unit stream from the start position corresponding to the service cycle to the end position in order to calculate the amount of data Rs to be read for each case. It is.

In addition, the position of the unit stream is determined at the position where the amount of data to be read is minimized, and there is no change in order in the subsequent process. For example, the position of the first unit stream is determined and the position of the second unit stream is determined. When the read data amount Rs is larger than the array of the first unit stream, the operation is stopped.

The above process is repeated until all of the unit streams included in one service period are selected.

S (0) = <us_1, us_2, us_3, ..., us_N>, the order in which the initial unit stream is arranged is us_n (n = 1, ...., N), which denotes a unit stream, and S (i-1 Select us_i and move the position from the first position to the Nth position to calculate the amount of data (Rs) that should be read from the disk in each case.

Then, the array of streams having the smallest amount of data Rs to be read from the disk is determined as S (i), and the value of i is increased by one. If i <= N, that is, the position is selected up to the last unit stream. If not, repeat the above process for the next unit stream, and if i> N, i.e., if the position is selected up to the last unit stream, then the whole process is terminated, and the unit stream is processed in the order of S (N) determined in the above process. To save it to disk.

This is a method that can reduce the amount of calculation of the multi-dimensional scalable video data storage method according to the present invention, and it is preferable to use it when an increase in speed is required even if it is not an optimal value.

4 is a diagram illustrating a unit stream configuration of a scalable video stream having spatial scalability and temporal scalability according to the present invention. As shown in the figure, spatial scalability such as the size of a video screen is expressed in levels (units) such as 1, 2, 3, etc., and temporal scalability such as frame rate (such as 1, 2, 3, etc.) Unit).

Here, the GOP, which is a unit capable of independently playing video, that is, a picture set, includes one or more unit streams, and in order to increase temporal scalability and spatial scalability, one or more unit streams must be further included.

For example, GOP ^k (2, 2) is a unit stream u ^k (1,1) with a level of temporal scalability 1, spatial scalability 1, and a level of temporal scalability 1, spatial scalability 2 Unit stream u ^k (1,2), unit stream u ^k (2,1) with temporal scalability 2, spatial scalability 1, and temporal scalability 2, spatial scalability 2 u ^k (2, 2).

Here, further to include the stream unit u ^k (3,1) and, u ^k (3,2) in order to convert the temporal scalability ^{3, GOP k (3, 2} ) having a level of spatial scalability 2, In order to read GOP ^k (3, 2) from the disk, all the unit streams below the unit stream u ^k (3,2) must be read.

In addition, in order to convert to GOP ^k (2, 3) having a level of temporal scalability 2 and spatial scalability 3, the unit stream u ^k (1, 3) and u ^k (2, 3) must be further included. to read from the disk GOP ^k (2, 3) stream units in the lower stream of the unit u ^k (2,3) they must read everything. .

FIG. 5 is a view schematically illustrating a storage state of general video data and a procedure of storing video data in a method of storing multidimensional scalable video data according to the present invention, and FIG. 6 is a diagram illustrating a multidimensional scalable video data according to the present invention. Figure 1 shows an embodiment to which the storage method is applied. As shown in the figure, a method of storing multidimensional scalable video data according to the present invention is as follows.

First, when the original video data stored in decoding order is broken into streaming service cycle units, each unit stream is grouped into units of spatial scalability level.

Here, u ^k (1,1), u ^k (1,2) and u ^k (1,3), which are unit streams having a spatial scalability level of 1, are grouped into a set, and are bound to a spatial scalability level 3 I can confirm that there is.

Here, in (a), CASE 1 reading a unit stream of spatial scalability 1, temporal scalability 2, CASE 2 reading a unit stream of spatial scalability 2, temporal scalability 1, spatial scalability 2, temporal scalability The following description will be made by dividing the case of CASE 3 reading a unit stream of 2.

In case of CASE 1, a unit stream of u ^k (1,2) or less should be included. Therefore, when data is sequentially stored, u ^k (1,1), u ^k (2,1), u ^k (3 , 1), u ^k (2,1) should be read, but u ^k (2,1), u ^k (3,1) because only u ^k (1,1) and u ^k (1,2) should read ) Becomes unnecessarily read data.

In case of CASE 2, the unit stream of u ^k (2,1) or less must be read. Therefore, when data is stored sequentially, u ^k (1,1) and u ^k (2,1) must be read. There is no data.

In case of CASE 3, a unit stream of u ^k (2, 2) or less should be included. Therefore, when data is stored sequentially, u ^k (1,1), u ^k (2,1), u ^k (3 , 1), u ^k (2,1), u ^k (2,2) should be read, u ^k (1,1), u ^k (2,1), u ^k (2,1), u ^k ( Since only 2,2) is data to be read, u ^k (3,1) becomes unnecessarily read data.

Accordingly, in (b), it can be seen that the read data amount is arranged and stored in the multi-dimensional scalable video data storage method according to the present invention so that the amount of read data is the smallest, that is, the search operation is minimized according to the required scalability level.

Where u ^k (2,3), u ^k (1,3), u ^k (1,2), u ^k (1,1), u ^k (2,1), u ^k (2,1), u ^k (2,2), u ^k (3,1), u ^k (3,2), u ^k (3,3), and in the case of CASE 1, u ^k (1,2) or less As the unit stream must be read, only u ^k (1,1) and u ^k (1,2) are data to be read, and since there is no unit stream included in the middle, the read data amount can be reduced.

Also, in case of CASE 2 and CASE 3, since there is no unit stream included in the middle, the read data amount may be reduced, and even if the order is changed because they are arranged to be mixed, the streaming service may be performed by sorting the data sequentially using an index. Can be.

7 is a diagram illustrating an embodiment to which a method of storing multidimensional scalable video data according to the present invention is applied. As shown in the figure, the multi-dimensional scalable video data storage method according to the present invention is as follows.

In (a), CASE 1 reading a unit stream of spatial scalability 1, temporal scalability 2, CASE 2 reading a unit stream of spatial scalability 2, temporal scalability 1, and spatial scalability 2, temporal scalability 2 The description will be made by dividing into the case of CASE 3 that reads a unit stream.

In case of CASE 1, a unit stream of u ^k (1,2) or less should be included. Therefore, when data is sequentially stored, u ^k (1,1), u ^k (2,1), u ^k (3 , 1), u ^k (2,1) should be read, but u ^k (2,1), u ^k (3,1) because only u ^k (1,1) and u ^k (1,2) should read ) Becomes unnecessarily read data.

In case of CASE 2, the unit stream of u ^k (2,1) or less must be read. Therefore, if data is stored sequentially, u ^k (1,1) and u ^k (2,1) must be read. There is no data.

In case of CASE 3, a unit stream of u ^k (2, 2) or less should be included. Therefore, when data is stored sequentially, u ^k (1,1), u ^k (2,1), u ^k (3 , 1), u ^k (2,1), u ^k (2,2) should be read, u ^k (1,1), u ^k (2,1), u ^k (2,1), u ^k ( Since only 2,2) is the data to be read, u ^k (3,1) becomes the data to be read unnecessarily.

Accordingly, in (b), it can be seen that the read data amount is arranged and stored in the multi-dimensional scalable video data storage method according to the present invention so that the amount of read data is the smallest, that is, the search operation is minimized according to the required scalability level.

Where u ^k (2,3), u ^k (1,3), u ^k (1,2), u ^k (1,1), u ^k (2,1), u ^k (2,1), u ^k (2,2), u ^k (3,1), u ^k (3,2), u ^k (3,3) are arranged so that they overlap twice, and for CASE 1, u ^k (1,2 Since the following unit streams should be read, only u ^k (1,1) and u ^k (1,2) are data to be read, and since there is no unit stream included in the middle, the read data amount may be reduced.

Also, in case of CASE 2 and CASE 3, since there is no unit stream included in the middle, the read data amount may be reduced, and even if the order is changed because they are arranged to be mixed, the streaming service may be performed by sorting the data sequentially using an index. Can be.

In addition, the GOP, which is the k + 1th picture group, is similarly applied and stored in the disk as described above.

Although the preferred embodiments of the present invention have been described above by way of example, the scope of the present invention is not limited to such specific embodiments, and those skilled in the art are appropriate within the scope described in the claims of the present invention. It may be possible to change it.

As described above, the present invention having the configuration as described above prevents a search delay caused by a plurality of search operations when partially extracting a frame according to the level of scalability, thereby reducing the overhead of the disk. This reduces the overhead of the server, increases the throughput of memory and disk, and reduces the time to read excess data that is not required for decoding, thereby providing a seamless streaming service on the client's request. It can achieve such effects.

Claims (9)

A first step of dividing the multi-dimensional scalable video stream into unit streams according to the scalability level of adjusting the resolution; Arranging the unit streams in a single streaming service period unit; A third step of sequentially storing unit streams corresponding to each level of scalability to be read from the disk in each single streaming service cycle, rearranging the single stream to store video data on the disk to minimize read data to be read from the disk, and step; Method of storing multidimensional scalable video data comprising a. The method of claim 1, The rearrangement of the unit streams in the third step is an array in which read data is calculated according to all cases of the unit stream array, and the read data is the minimum. The method of claim 1, The rearrangement of the unit streams in the third step selects an arbitrary unit stream from the unit stream array, calculates read data according to each case while positioning the positions from the start point to the end point of the unit stream array, and calculates the read data. A method of storing multidimensional scalable video data, wherein the read data is a minimum array. The method of claim 3, When selecting an arbitrary unit stream, selecting one of the last unit stream in the first unit stream, wherein all of the unit streams of the unit stream array, characterized in that for selecting once each of the multi-dimensional scalable video data. The method of claim 1, And the read data may vary according to each streaming service period. The method of claim 1, The scalability level is Temporal scalability to adjust the frame rate of the video; Spatial scalability to resize the screen to play video; SNR scalability to adjust bit rate coding video data; Method for storing multidimensional scalable video data comprising a. The method of claim 1, The unit stream is a partial stream of data corresponding to a group of pictures (GOP), and is a minimum data block capable of generating a video having increased and decreased levels of scalability. The method of claim 1, When a plurality of picture groups are included in the streaming service period, the unit streams in each picture group may be mixed to minimize read data. The method of claim 1, Multi-dimensional scalable video data, characterized in that the re-arranged unit streams in the third step are indexed in a mixed state and stored on a disk, but when the streaming service is performed, the unit streams can be loaded in memory according to the index. Way of storage.

Images