CN104918058A - High-efficiency data flow transmission method - Google Patents
High-efficiency data flow transmission method Download PDFInfo
- Publication number
- CN104918058A CN104918058A CN201510273751.5A CN201510273751A CN104918058A CN 104918058 A CN104918058 A CN 104918058A CN 201510273751 A CN201510273751 A CN 201510273751A CN 104918058 A CN104918058 A CN 104918058A
- Authority
- CN
- China
- Prior art keywords
- data
- generation module
- data flow
- file
- zero
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
- H04N21/23103—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
- H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Disclosed in the invention is a high-efficiency data flow transmission method. The method comprises: a universal interface is provided for a file system of a multi-media server, thereby improving the throughput capacity of the data flow based on avoidance of data copying and context transformation. The file system includes an output registration service program that is used for providing all function instruction words pointing to the file system to realize a group of specified zero-copy interfaces. The interfaces contain: an initialized service program, which is used for establishing a necessary data structure; another service programs, which are used for verifying whether the specified file is suitable for zero-copy operation; another service program, which is used for locking a buffer at an interrupt level and returning to the existing data volume; and a service program, which is used for providing information related with the file access mode change.
Description
Technical field
The present invention relates to computerized multimedia server, specifically, relate to the technology such as transmitting data flow in this system expeditiously between each device from disk storage sub-system to network or image decoding card.
Background technology
Multimedia system is usually directed to the data flow representing highly dense multi-medium data in a large number, and they must transport through system with very high efficiency.A representational example is " video request program " service system.People have studied various method, try hard to make data by these systems expeditiously.These systems have many well-known Unix operating systems or its derived system as their operating system, as usual realize can seeing in the client server environment of content of multimedia delivery system at those.
But the intrinsic wretched insufficiency part of this system based on Unix wherein transmits the mechanism that data adopt not make efficiency optimization as necessary in the system relating to very complicated multimedia data stream.
Therefore, industry is badly in need of a kind of method of efficient transmission data flow, and current market situation is improved.
Summary of the invention
In order to overcome the technological deficiency existed in above-mentioned prior art, the object of the present invention is to provide a kind of method of efficient transmission data flow.
In order to realize foregoing invention object, technical scheme of the present invention is as follows:
A kind of method of efficient transmission data flow, it for performing zero-copy function with the file system support be pre-existing in a data streaming file in multi-medium data streaming system, described multi-medium data streaming system comprises: an existing file system in the different existing file systems of multiple each support one group of zero-copy function interface, a VCS that a generation module is connected with a line module, a mass storage device be connected with shared storage, a user's set, a multimedia system framework background process and a device driver be configured between described user's set and line module,
Described method comprises the following steps: to set up a universal document system interface be connected from described multiple different existing file system; And call described universal document system interface, a control system, to realize the zero-copy transmission of the described data flow from described generation module to described line module.
Further, described method also comprises the following steps: to send a funcall from described generation module to described output enrolled for service, to obtain the described function descriptors that described generation module can be used to call described kernel extensions.
Further, described method also comprises the following steps: to call described generation module with described VCS, verifies whether described data streaming file meets the requirement of described zero-copy function to make described generation module.
Wherein, described verification step comprises and carries out initialization and inspection some parameters relevant to described data streaming file to data structure.
Preferably, described method also comprises the following steps: to call the expansion of described kernel services from described generation module, to obtain supporting that therefrom the mark of the buffer of data is transmitted in conclusion border, thus determine the position of described data flow in described mass storage device and can be retrieved in interruption linguistic context.
Preferred, described method also comprises the following steps:, from described generation module to described kernel services expansion transmission designator, to indicate described system and can prepare the data that need from described mass storage device transmit through described shared storage in buffer corresponding with described data streaming file at any time.
High efficiency of carrying out conventional subsystem transmits some other problem that multi-medium data stream information occurs, make to this invention takes corresponding innovation measure, comprise: find and lock cache device in interrupt class, prefetching information is transmitted to background process, got in advance before needing file data buffer in interrupt class, with being written in parallel to device, serialization is carried out to file, and more put it briefly, be provided in the universal document system interface supporting zero-copy in VCS framework, to make the transmission of data more efficient.
Embodiment
Below in conjunction with specific embodiment, further detailed description explanation is done to technical scheme of the present invention.
Present embodiments provide zero-copy or VCS subsystem, the mated condition of each part in its function and a more traditional system.Data output is mainly used for setting up the connection between the such image decoding card of disc driver and file system and such as ADEC.Put it briefly, delivery outlet provides the interconnection in a pipeline fashion to generator and user's set.Once connection establishment, data output, by controlling to provide a connection to VCS, is used for by the data flowing between the direct control documents system of VCS and communicator.
The general function of image connexon system be control in the best way multimedia system framework (MMFS), between generation module and line module data flowing.Line module can be such as one and comprise such as ADECNAR and be substantially in the image decoder that streamline exports the corresponding A/D EC adapter of that side.Generation module can be one such as partly can be embodied in disk drive system in multiple conventional disc driver or a RAID array and the such storage subsystem of file system.It can be have the intermediation between the such file system of the such as file system of existing common structure and VCS that generation module plays another.VCS itself has the interface of a fabric anomaly data flowing, and therefore generation module plays in fact the data that respective calls is asked with acquisition are carried out in the request of response from VCS intermediation to file system.
Kernel extensions is obviously be among operating system nucleus, and background process performs in application space.Which illustrate why control flow be from VCS to generator block to kernel extensions again to background process, VCS, generator block and kernel extensions all in kernel, sensible user class representated by the background process in application space always.
Basic object of the present invention realizes required multimedia data transmission and do not need to carry out copy to data taking expense.Knowing, data can be transmitted between each device when data not being taken into memory.
In image server environment, possibility is interested is that data are taken into memory, because if there are two users to wish to play same program, so these multiple requests can meet (if but directly transmit identical data between the devices, so these data just must transmit twice physically) with the single copy in memory.Therefore, can wish this user to access in the image server environment of identical data, can sharp memory-aided damping characteristics be significant.Model of the present invention is based on such imagination: data will be taken into memory and hope obtains individual data copy, and selected scheme is make file system provide this single copy substantially.
File system is run traditionally in process linguistic context.In this linguistic context, the traditional approach transmitting data in file system carries out " reading " system call from a user class program to kernel, obtains the particular data scope of reading in the file of a kernel cache device from dish, and then copy it to user cache device.On the other hand, if need data to write on network, so just send " a writing " system call, data are oppositely sent to kernel from user memory, then send network equipment to from kernel cache device.
The problem that the present invention studies relates to the model that image connexon system can be supported.Specifically, it is desirable that such a system, operate in the mode that can limit the expense shared by CPU overhead and copies data in the environment (AIX of such as IBM Corporation) TM (operating system) that it can derive from Unix or Unix.The control flow that VCS performs between generation module and line module mostly occurs in interruption linguistic context.But, in the environment of the operating system of Unix and Unix derivation, to having obvious restriction in the operation interrupting performing in environment (different from the process linguistic context that file system is run) wherein.VCS of the present invention replaces traditional writing model with a model carried out data transmission in kernel completely.
In this, with an example, the situation adopting Zero-copy method proposed by the invention to transmit data can be described.Between generation module and line module, establish a streamline once VCS and start to transmit data flow, VCS will send a request to generation module, requires that it obtains a data buffer.So generation module incoming file system, execution is searched, and determines the position of data buffer, is then provided in the addressable mark of kernel linguistic context to this buffer, is given to line module.Therefore, not actual copy data, and be to provide the addressing benchmark to data.
Specifically, code mainly performs in generation module.Therefore, after image subsystems starts, the kernel of generation module loading of operating system, after generation module loads, just call in the output service provided by kernel extensions, this is an enrolled for service substantially.The appropriate address of this kernel extensions in order to each function of supporting zero-copy function to provide is pointed out in the service of generation module request kernel extensions.After process starts to load with generation module, namely register the file system zero-copy supporting interface supported of kernel extensions.
It is noted that an important feature of the present invention there is provided the general-purpose interface supporting this zero-copy function.According to the present invention, owing to providing one group of more general kernel services, therefore can be used by the various different file system in zero-copy framework.These file system are registered by generation module.Specifically, this of respond module calls, and kernel extensions returns one group of function descriptors to generation module, can carry out various funcall after making generation module to kernel extensions.Therefore, generation module carries out some initialization by its data structure, and parametric test is carried out to the file sending it to, with ensure this file actual be a multimedia file from multimedia system framework instead of one from the file of such other what file system of such as magazine file system (JFS).
Before data transmit actual beginning, generation module will call in existing multimedia system framework (specifically MMFS kernel extensions), extract relevant to allow the information of the data therefrom file of retrieve data in this interruption linguistic context later.
As previously described, usually in the environment of Unix or Unix derivation, most of file system operation, as from file system sense data or by data writing in files system, all occurs in process linguistic context.Data can be transmitted in file system and must obtain all information needed for transmitting, such as relevant document, support that the information of the buffer of data etc. is transmitted in therefrom conclusion border.The respective data structures of such as node and indirect block and so on is traditionally in process linguistic context instead of addressable in interruption linguistic context.
Therefore, user's set has in fact completed the transmission of the current buffer in shared storage.Once make this notice transmitted, the any locking indicating this buffer using or other information all must upgrade, in addition, so another prefetch message sends to background process from generation module by the path of kernel extensions, to carry out look ahead (position namely will transmitted hereof, the data imported into the last time in logic in the firm buffer discharged are adjacent) of next buffer before user's set needs.Like this, system just keeps so a kind of state, and before user's set needs, data have just existed in shared storage.
Test, determine whether that other buffers leave the need corresponding with file and transmit data.In fact, if also do not reach the terminal of data, process just loops back square frame.If test shows that all these data transmit all, then process terminates, and sends return messages, represents that data transmit and completes.
Although in conjunction with specific embodiments more of the present invention, the invention has been described; but those skilled in the art are appreciated that; wherein no matter various modifications can be can carried out in form or in detail according to Spirit Essence of the present invention, therefore all scope of patent protection of the present invention should be belonged to.
Claims (6)
1. one kind is efficiently transmitted the method for data flow, it for performing zero-copy function with the file system support be pre-existing in a data streaming file in multi-medium data streaming system, described multi-medium data streaming system comprises: an existing file system in the different existing file systems of multiple each support one group of zero-copy function interface, a VCS that a generation module is connected with a line module, a mass storage device be connected with shared storage, a user's set, a multimedia system framework background process and a device driver be configured between described user's set and line module, it is characterized in that:
Described method comprises the following steps: to set up a universal document system interface be connected from described multiple different existing file system; And call described universal document system interface, a control system, to realize the zero-copy transmission of the described data flow from described generation module to described line module.
2. the method for a kind of efficient transmission data flow according to claim 1, it is characterized in that, described method also comprises the following steps: to send a funcall from described generation module to described output enrolled for service, to obtain the described function descriptors that described generation module can be used to call described kernel extensions.
3. the method for a kind of efficient transmission data flow according to claim 1 and 2, it is characterized in that, described method also comprises the following steps: to call described generation module with described VCS, verifies whether described data streaming file meets the requirement of described zero-copy function to make described generation module.
4. the method for a kind of efficient transmission data flow according to claim 3, is characterized in that, described verification step comprises and carries out initialization and inspection some parameters relevant to described data streaming file to data structure.
5. the method for a kind of efficient transmission data flow according to claim 3, it is characterized in that, described method also comprises the following steps: to call the expansion of described kernel services from described generation module, to obtain supporting that therefrom the mark of the buffer of data is transmitted in conclusion border, thus determine the position of described data flow in described mass storage device and can be retrieved in interruption linguistic context.
6. the method for a kind of efficient transmission data flow according to claim 5, it is characterized in that, described method also comprises the following steps:, from described generation module to described kernel services expansion transmission designator, to indicate described system and can prepare the data that need from described mass storage device transmit through described shared storage in buffer corresponding with described data streaming file at any time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510273751.5A CN104918058A (en) | 2015-05-26 | 2015-05-26 | High-efficiency data flow transmission method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510273751.5A CN104918058A (en) | 2015-05-26 | 2015-05-26 | High-efficiency data flow transmission method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104918058A true CN104918058A (en) | 2015-09-16 |
Family
ID=54086702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510273751.5A Pending CN104918058A (en) | 2015-05-26 | 2015-05-26 | High-efficiency data flow transmission method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104918058A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0853412A2 (en) * | 1996-12-02 | 1998-07-15 | International Business Machines Corporation | System method for efficiently transferring datastreams in a multimedia system |
| JP2007318792A (en) * | 1994-05-13 | 2007-12-06 | Starsight Telecast Inc | Television schedule information transmission and utilization system and process |
| CN101557420A (en) * | 2009-03-31 | 2009-10-14 | 北京航空航天大学 | Realization method of high-efficiency network communication of a virtual machine monitor |
| CN102291298B (en) * | 2011-08-05 | 2014-10-22 | 曾小荟 | Efficient computer network communication method oriented to long message |
-
2015
- 2015-05-26 CN CN201510273751.5A patent/CN104918058A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007318792A (en) * | 1994-05-13 | 2007-12-06 | Starsight Telecast Inc | Television schedule information transmission and utilization system and process |
| EP0853412A2 (en) * | 1996-12-02 | 1998-07-15 | International Business Machines Corporation | System method for efficiently transferring datastreams in a multimedia system |
| CN101557420A (en) * | 2009-03-31 | 2009-10-14 | 北京航空航天大学 | Realization method of high-efficiency network communication of a virtual machine monitor |
| CN102291298B (en) * | 2011-08-05 | 2014-10-22 | 曾小荟 | Efficient computer network communication method oriented to long message |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1093660C (en) | System method for efficiently transferring datastreams in multimedia system | |
| US10120586B1 (en) | Memory transaction with reduced latency | |
| KR102520039B1 (en) | System and method for supporting energy and time efficient content distribution and delivery | |
| US8762480B2 (en) | Client, brokerage server and method for providing cloud storage | |
| US20020168966A1 (en) | I/O unit emulation | |
| US20080147909A1 (en) | Remote USB protocol for a heterogeneous system | |
| CN104603739A (en) | Block-level access to parallel storage | |
| US9104813B2 (en) | Software installation method, apparatus and program product | |
| JPH09224044A (en) | Distribution system | |
| US7069305B2 (en) | Computer system and a data transfer method thereof using remote direct memory access | |
| CN108205482B (en) | File mount restoration methods | |
| CN107817962B (en) | Remote control method, device, control server and storage medium | |
| US20050193230A1 (en) | Metadata access during error handling routines | |
| CN111164570A (en) | System and method for creating a virtual disk image for use with a remote computer | |
| US11868819B2 (en) | System and method for constructing filter graph-based media processing pipelines in a browser | |
| US7822040B2 (en) | Method for increasing network transmission efficiency by increasing a data updating rate of a memory | |
| CN104918058A (en) | High-efficiency data flow transmission method | |
| CN1625734A (en) | System and method of streaming data to computer in a network | |
| CN116644022A (en) | Data processing node and method | |
| US10063256B1 (en) | Writing copies of objects in enterprise object storage systems | |
| US20050050252A1 (en) | Information processing apparatus | |
| CN116932451A (en) | Data processing method, host and related equipment | |
| US20080140669A1 (en) | Dedicated storage and background backup of stored contents | |
| CN101510146A (en) | Virtual space establishing method, apparatus and system based on independent redundant magnetic disc array | |
| US6434635B1 (en) | Methods, apparatus, and computer program product for data transfer using a scatter-gather list |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150916 |
|
| WD01 | Invention patent application deemed withdrawn after publication |