JP3844485B2 - Internet live broadcast system for mobile phone, Internet live broadcast method for mobile phone, streaming data creation and distribution program, and recording medium - Google Patents

Description

  The present invention relates to streaming broadcasting of moving images with sound to mobile phone terminals that can be connected to the Internet.

  With the improvement in performance of mobile phone terminals, it is possible to view video with audio, playback of video with audio recorded in advance, and use videophone and videophone systems to exchange each other's audio and video with the other party in real time. As a result, services that deliver video with audio in real time have been put into practical use.

  In addition, mobile phone carriers offer services that offer flat-rate Internet packet communication costs, such as video files with audio that require a significant amount of data transfer compared to sending and receiving emails and browsing websites. Can be easily sent and received.

Therefore, in order to provide a new service using a mobile phone, as a prior invention of the present invention, live video that does not include audio is streamed in real time to the mobile phone, and at the same time, bidirectional communication using character information is performed in real time. The applicant has applied for a patent related to a mobile phone chat service system.
Japanese Patent Application No. 2004-180113

  If you want to watch a video with audio on a mobile phone and play it back in advance after recording the desired video with audio, you will need to make preparations that require time and effort that cannot be ignored. In addition, there is a problem that real-time viewing of moving images is impossible. Furthermore, since it is necessary to temporarily store the moving image file in the storage device of the mobile phone terminal, the length of the moving image is limited by the capacity of the storage device, and there is a problem that only a moving image of about several minutes can actually be viewed. is there.

  In addition, since videophones are not covered by the Internet packet communication fee flat-rate service, there is a problem that a high charge is charged in proportion to the moving image viewing time.

Here, the main objects to be solved by the present invention are as follows.

  The first object of the present invention is a mobile phone terminal that can be connected to the Internet, and performs live streaming for live audio, video, and text information so that they can be streamed in real time synchronously without interruption for a long time. It is to provide a broadcasting method and system.

  A second object of the present invention is a mobile phone terminal that can be connected to the Internet. Even if the communication speed of the mobile phone is reduced, the bit rate and the frame rate are reduced and the multi-bit rate streaming delivery is stable. It is to provide an Internet live broadcast method and system for mobile phones.

  A third object of the present invention is to provide an Internet live broadcasting method and system for mobile phones that can be easily applied to video-on-demand streaming distribution viewed on a mobile phone terminal connectable to the Internet.

  A fourth object of the present invention is to provide an Internet streaming broadcasting method for mobile phones, which is compatible with the Internet packet communication flat-rate service by using Internet packet communication, and is capable of viewing a moving image with sound for a long time with a small communication cost. To provide a system.

  Other objects of the present invention will become apparent from the specification, drawings, and particularly the description of each claim in the claims.

  First, in the system of the present invention, the streaming data creation server includes live source capturing means for capturing live audio, video and text information, and compressing and converting the captured live audio, video and text information in real time, Streaming data generating means for generating streaming data suitable for each model of the above and streaming data distribution means for immediately distributing the generated streaming data to an Internet broadcast server via the Internet communication network, the Internet broadcast The server receives streaming data that is constantly distributed from the streaming data creation server via the Internet communication network one by one, stores the streaming data in a storage device, and stores the stored data stored in the storage device. Streaming data management means for managing the game data, menu management data distribution request receiving means for receiving a menu management data distribution request from the mobile phone terminal via the Internet communication network, and the mobile communication terminal via the Internet communication network. Menu management data distribution means for updating the menu management data requested from the telephone terminal and distributing it to the portable telephone terminal, and streaming content for receiving a streaming content distribution request from the portable telephone terminal via the Internet communication network Distribution request receiving means, streaming data distribution means for distributing streaming data requested from the mobile phone terminal to the mobile phone terminal via the Internet communication network, distribution of streaming content to the mobile phone terminal Streaming content distribution management means for managing the content, and the mobile phone terminal requests menu management data necessary for displaying a menu screen to the Internet broadcast server via the Internet communication network. Menu management data distribution requesting means, menu management data receiving means for receiving the requested menu management data from the Internet broadcast server via the Internet communication network, and a menu displayed based on the received menu management data A streaming content distribution requesting unit for selecting streaming content from a screen and requesting the Internet broadcast server to start distributing the selected streaming content via the Internet communication network; Streaming data storage and storage means for storing and storing streaming data continuously distributed from the Internet broadcast server via the communication network in the storage device one by one, live audio, video and character information from the stored and stored streaming data Streaming data restoring means for restoring various data, and streaming viewing means for reproducing live audio, video and text information synchronously in real time without interruption for a long time using the restored various data. The characteristic composition means is taken.

  On the other hand, in the method of the present invention, the streaming data creation server first captures live audio, video, and text information by live source capture processing, and then performs streaming data creation processing for the live audio, video, and text information. Compresses and converts information in real time, creates streaming data suitable for each mobile phone model, and then immediately delivers the streaming data to the Internet broadcast server via the Internet communication network by streaming data distribution processing On the other hand, the Internet broadcast server receives the streaming data received from the streaming data creation server via the Internet communication network one by one by the streaming data storage and accumulation process, and stores and stores it. In addition, the streaming data management process manages the stored and stored streaming data, and the menu management data distribution request reception process receives the menu management data distribution request from the mobile phone terminal via the Internet communication network. The menu management data distribution process updates the menu management data requested from the mobile phone terminal via the Internet communication network, distributes it to the mobile phone terminal, and the streaming content distribution request reception process performs the Internet communication. A streaming content delivery request is received from the mobile phone terminal via the network, and streaming data requested from the mobile phone terminal is delivered to the mobile phone terminal by streaming data delivery processing. The distribution of streaming content to the mobile phone terminal is managed by the content distribution management process. On the other hand, the mobile phone terminal performs the menu management data distribution request process to the Internet broadcast server via the Internet communication network. Requesting the menu management data necessary to display the menu screen, and receiving the requested menu management data from the Internet broadcast server via the Internet communication network by the menu management data receiving process, and streaming In the content distribution request process, the streaming content is selected from the menu screen displayed based on the received menu management data, and the selected content is transmitted to the Internet broadcast server via the Internet communication network. Streaming content is requested to start distribution, streaming data distributed from the Internet broadcast server via the Internet communication network is stored and stored in the storage device one by one by streaming data storage and storage processing, and streaming data restoration processing is Various data such as live audio, video, and text information is restored from streaming data, and the restored various data is reproduced by streaming playback processing and continuously playing the audio, video, and text information at the same timing. Using this, a characteristic configuration technique is adopted in which live audio, video, and text information are reproduced and output in real time synchronously without interruption for a long time.

  On the other hand, in the streaming data creation / delivery program of the present invention, a live source import processing procedure for capturing live audio, video and character information into the streaming data creation server, and the captured live audio, video and character information are compressed in real time. Streaming data creation processing procedure for converting and creating streaming data suitable for each mobile phone model, and streaming data delivery processing procedure for immediately delivering the created streaming data to the Internet broadcast server via the Internet communication network And a characteristic configuration procedure of sequentially executing.

  The recording medium of the present invention employs a characteristic configuration procedure in which a practical procedure in the streaming data creation and distribution program of the present invention is recorded in a computer-readable manner.

  More specifically, in solving the problem, the object is achieved by adopting a novel characteristic configuration method, means, procedure and procedure from the superordinate concept to the subordinate concept that the present invention enumerates next. To be achieved.

That is, the first feature of the system of the present invention is that a streaming data creation server for creating and distributing streaming data obtained by compressing and converting live audio, video and character information in real time, and the streaming received via the Internet communication network. Stores and stores data, and restores from the Internet broadcast server that distributes the stored and stored streaming data in response to a distribution request from the mobile phone terminal, and the streaming data received from the Internet broadcast server via the Internet communication network sound, a mobile phone Internet live broadcasting system which is constructed with a streaming reproduction output enabled mobile phone in real time synchronously without interruption long video and character information, the streaming data creation server Is the live audio acquired by the audio acquisition device, the live video shot by the video shooting device, the live audio stored in advance in the streaming server and the text information that is the content of the subtitles displayed in synchronization with the live audio A live source capturing means that captures the image, and the captured live audio is compressed and processed as digital audio data so that the captured live audio can be played back as a ringtone that is unique to a mobile phone terminal in synchronization with the video data in real time. The live video is divided into a plurality of images in synchronization with the audio data in real time, and a still image is extracted. The digital video data is further compressed and processed as digital video data that can be played back on the mobile phone terminal. phone machine for coupling said digital video data and the character information Streaming data creating means for creating streaming data suitable for each, and streaming data delivery means for immediately delivering the created streaming data to the Internet broadcast server via the Internet communication network. The creation means includes a streaming data storage and storage means for receiving the streaming data constantly delivered from the streaming data creation server via the Internet communication network, and storing and storing the streaming data in a storage device. A streaming data management means for managing the streaming data, and a menu management data distribution request for receiving a menu management data distribution request from the mobile phone terminal via the Internet communication network Receiving means, updating the menu management data requested from the mobile phone terminal via the Internet communication network, and delivering the menu management data distribution means to the mobile phone terminal, via the Internet communication network Streaming content delivery request receiving means for receiving a delivery request for streaming content from a mobile phone terminal, and streaming data delivery means for delivering streaming data requested from the mobile phone terminal to the mobile phone terminal via the Internet communication network And streaming content distribution management means for managing distribution of streaming content to the mobile phone terminal, wherein the streaming data is divided for each unit time and each data size is the mobile phone end A size that can be accommodated in the storage area of the inner and includes a video data and audio data composed of still image data independently of each other, the cellular phone terminal, via the Internet communication network, the Internet broadcast Menu management data distribution request means for requesting menu management data necessary for displaying a menu screen to the server, and receiving the requested menu management data from the Internet broadcasting server via the Internet communication network. Selecting a streaming content from the menu management data receiving means to be performed and a menu screen displayed based on the received menu management data, and delivering the selected streaming content to the Internet broadcast server via the Internet communication network Request to start Streaming data distribution requesting means, streaming data storage and storage means for storing and storing streaming data continuously distributed from the Internet broadcast server in the storage device one by one via the Internet communication network, and the stored and stored streaming data Streaming data restoration means for restoring various data such as live audio, video and character information , an audio output unit for reproducing live audio in real time using the restored various data, and the video and character information A streaming viewing unit having a video output unit that plays back synchronously with the audio, and the audio output unit uses a plurality of audios that are alternately used so that continuous audio data can be played back without interruption for a long time. A mobile phone characterized by having a playback channel Is adopting the configuration of the Internet live broadcasting system.

The second feature of the system of the present invention is that the streaming data creation means of the streaming data creation server in the first feature of the present invention system synchronizes the captured live audio, video and character information every unit time. , And compress and convert the separated live audio into an audio file in a format that can be output by the audio output function that is an essential function of the mobile phone terminal. Each of the extracted still images is compressed and converted into independent still image files in a format that can be displayed by the still image display function provided as a basic function of the mobile phone terminal, and the compression conversion is performed. Selected audio files, a series of still images files that have been compressed and converted, and the character information file Management information indicating information such as the size of these files and the like, and main streaming data configured so that streaming viewing at a low bit rate and a low frame rate can be performed alone, and the rest of the compressed series of still image files. And management information indicating information such as the size of these files, and sub-streaming data configured to enable streaming viewing at a high bit rate and a high frame rate only together with data stored in the main streaming data. The streaming data distribution means of the Internet broadcast server according to the first feature of the system of the present invention is configured to generate divided and multi-bit rate streaming data for each model of the mobile phone. The communication speed of the phone terminal If the amount of data is high, the main streaming data and the sub-streaming data, which are all of the multi-bit rate streaming data, are distributed to the mobile phone terminal to perform high bit rate streaming distribution. When the speed is low, the system is configured to perform low bit rate streaming distribution by distributing only the main streaming data, which is a part of the multi-bit rate streaming data, to the mobile phone terminal. When the streaming viewing means of the mobile phone terminal according to the first feature of the present invention outputs a video by sequentially displaying a series of still image files obtained by the streaming data restoration means every unit time, The bit rate at the time of reception It is configured to play and output audio and video using multi-bit rate streaming data synchronously by dynamically changing and setting the frame rate for displaying the series of still images according to the height. Is adopting the configuration of the Internet live broadcast system for mobile phones.

  The third feature of the system of the present invention is that the streaming viewing means of the mobile phone terminal in the first or second feature of the system of the present invention divides the screen display of the mobile phone terminal into two parts, and live 2. The Internet live broadcasting system for mobile phones according to claim 1, characterized in that the video is stream-played and output in real time, and the other is configured to play back and display subtitles and advertisements as text information or still images. The configuration is adopted.

A fourth feature of the system of the present invention is that the streaming data storage and storage means of the Internet broadcast server in the first, second or third feature of the present invention system stores streaming data of existing content that has already been subjected to compression conversion processing. Is also configured to store and store the Internet live broadcast system for mobile phones, which is characterized in that it is configured.

On the other hand, the first feature of the method of the present invention is that streaming data created in real time by a streaming data creation server from live audio, video and text information is distributed to a mobile phone terminal via an Internet communication network and an Internet broadcast server. The mobile phone Internet live broadcasting method for mobile phone Internet live streaming and outputting the live audio, video and text information synchronously in real time without interruption for a long time. Live audio acquired by the audio acquisition device by the source capturing process, live video captured by the video imaging device, the live audio stored in the streaming server in advance and the content of the subtitles displayed in synchronization with the live audio, Character information Uptake, then the live audio captured the compress processed as a renewable digital audio data as ringtones a mobile phone-specific features in synchronization with the video data in real time, and live video captured the Are divided into a plurality of images in synchronization with the audio data in real time, extracted as still images, and compressed and processed as digital video data reproducible for the mobile phone terminal, and further, the digital audio data and the digital video The data is combined with the character information in a method suitable for each mobile phone model, each data size can be stored in the storage area in the mobile phone terminal, and the video data is composed of still image data independent from each other create a streaming data to be followed by, by the streaming data delivery process The streaming data is immediately distributed to the Internet broadcasting server via the Internet communication network, while the Internet broadcasting server receives the streaming data from the streaming data creation server via the Internet communication network by streaming data storage and accumulation processing. Streaming data is received and stored one by one, and the stored and accumulated streaming data is managed by a streaming data management process, and from the mobile phone terminal via the Internet communication network by a menu management data distribution request receiving process. A menu management data distribution request is received, and the menu management data distribution process updates the menu management data requested from the mobile phone terminal via the Internet communication network. Delivered to a mobile phone terminal, received a streaming content delivery request from the mobile phone terminal via the Internet communication network by a streaming content delivery request reception process, and requested from the mobile phone terminal by a streaming data delivery process. The streaming data is distributed to the mobile phone terminal, and the streaming content distribution management process manages the distribution of the streaming content to the mobile phone terminal. On the other hand, the mobile phone terminal performs the menu management data distribution request process, Requesting the menu management data necessary for displaying the menu screen to the Internet broadcast server via the Internet communication network, and through the Internet communication network by the menu management data receiving process From the Internet broadcast server, the requested menu management data is received, and streaming content distribution request processing is performed to select the streaming content from the menu screen displayed based on the received menu management data, via the Internet communication network, The Internet broadcast server is requested to start distribution of the selected streaming content, and streaming data distributed from the Internet broadcast server via the Internet communication network is stored in the storage device through the streaming data storage and accumulation process. Store and store, restore various data such as live audio, video and text information from the streaming data by streaming data restoration processing, and the audio by streaming viewing processing By continuously playing video and text information at the same time, live audio, video and text information can be reproduced and output in real time synchronously for a long time using the restored data. It is in the configuration adoption of the Internet live broadcasting method for mobile phones.

A second feature of the method of the present invention is that the streaming data creation processing of the streaming data creation server in the first feature of the method of the present invention described above captures the live audio, video, and character information captured in units of unit time. , And compress and convert the separated live audio into an audio file in a format that can be output by the audio output function that is an essential function of the mobile phone terminal. Each of the extracted still images is compressed and converted into independent still image files in a format that can be displayed by the still image display function provided as a basic function of the mobile phone terminal, and the compression conversion is performed. Audio files, a series of still images that have been compressed and converted, selected at regular intervals, the character information file, Management information indicating information such as the file size, main streaming data configured to be capable of streaming viewing at a low bit rate and a low frame rate, and the rest of the compressed series of still image files and Management information indicating information such as the size of these files is stored and divided into sub-streaming data that can be streamed at a high bit rate and a high frame rate only together with the data stored in the main streaming data. Then, multi-bit rate streaming data is created for each model of the mobile phone, and the streaming data distribution processing of the Internet broadcast server in the first feature of the method of the present invention is sufficient for the communication speed of the mobile phone terminal. If it is higher, the multi-bit rate Main streaming data and sub-streaming data, all of which are streaming data, are distributed to the mobile phone terminal to perform high bit rate streaming delivery. On the contrary, when the communication speed of the mobile phone terminal is low, the multi-bit Low bit rate streaming delivery is performed by delivering only main streaming data, which is a part of rate streaming data, to the mobile phone terminal, and the streaming viewing process of the mobile phone terminal according to the first feature of the method of the present invention is performed. , When outputting a video by sequentially displaying a series of still image files obtained by the streaming data restoration process every unit time, the series of still images according to the bit rate at the time of reception Dynamically change the frame rate to display The configuration adopts the configuration of the Internet live broadcast method for mobile phones, characterized in that, by setting, audio and video based on multi-bit rate streaming data are reproduced and output synchronously.

  A third feature of the method of the present invention is that the streaming viewing process of the mobile phone terminal in the first or second feature of the method of the present invention divides the screen display of the mobile phone terminal into two parts, and live video Is used for streaming reproduction output in real time, and on the other hand, subtitles and advertisements are displayed and output as text information or still images.

According to a fourth feature of the method of the present invention, the streaming data storage / accumulation processing of the Internet broadcast server according to the first, second or third feature of the method of the present invention described above is the streaming data of existing content that has already undergone compression conversion processing. The Internet live broadcasting method for mobile phones, which is characterized in that it also stores and accumulates.

On the other hand, the first feature of the streaming data creation / distribution program of the present invention is that streaming data for creating and delivering streaming data, both of which have the basic functions of a computer and in which live audio, video and character information are compressed and converted in real time A creation server, an Internet broadcast server that stores and stores the streaming data received via the Internet communication network, and distributes the stored and stored streaming data in response to a distribution request from the mobile phone terminal, and the Internet broadcast server A mobile phone terminal for reproducing and outputting live audio, video and text information restored from the streaming data received via the Internet communication network, and an Internet live broadcast system for mobile phones constructed with A streaming data creation / distribution program applied to the streaming data creation server, wherein a live source capture processing procedure for capturing live audio, video and text information into the streaming data creation server, and the captured live audio and video In addition, a streaming data creation processing procedure for compressing and converting character information in real time and creating streaming data suitable for each mobile phone model, and the created streaming data immediately to the Internet broadcast server via the Internet communication network A streaming data distribution processing procedure to be distributed, and the streaming data creation processing procedure includes a procedure for synchronously dividing the captured live audio, video and character information every unit time, The procedure to compress and convert the cut live audio into an audio file in a format that can be output by the audio output function that is an essential function of the mobile phone terminal, and on the other hand, extract still images from the divided live video for each unit time Each of the extracted still images is compressed and converted into independent still image files in a format that can be displayed by the still image display function that the mobile phone terminal has as a basic function, and the compression conversion is performed. Low-bit-rate audio files, a series of compressed and converted still image files selected at regular intervals, the character information file, the total number of these files, and management information indicating the size of each file. Main streaming data that can be streamed at a low frame rate by itself, and the compressed series Stores management information indicating the rest of still image files and information such as the size of these files, and is configured to enable streaming viewing at a high bit rate and a high frame rate only together with the data stored in the main streaming data. Streaming data and multi-bit rate streaming data are created for each model of the mobile phone terminal, and the mobile phone terminal receives the streaming data from the streaming server and is included in the streaming data The audio data, the still image file, and the character information are restored, and the continuous audio data included in the audio output unit in the mobile phone terminal is used as a plurality of audio reproduction channels that are alternately used so that they can be reproduced without interruption for a long time. And play Tomega certain file and the text information in the configuration adopting the voice and synchronously processing means for reproducing is sequentially executed streaming data generating distribution program comprising.

In addition, a first feature of the recording medium of the present invention is a streaming data creation / delivery program that is recorded in a computer-readable manner as a procedure for executing each processing procedure in the first feature of the streaming data creation / delivery program of the present invention. The configuration of the recorded recording medium is employed.

A mobile phone terminal that can be connected to the Internet is capable of streaming live audio, video and text information synchronously in real time for a long period of time within the limited computing power, storage capacity and communication speed. Demonstrate the effect.
Also, since each frame constituting the video is a still image independent from each other, even if the data of one frame is destroyed due to a communication accident or the like, it is like a moving image of the so-called MPEG series. It exhibits an excellent effect that it does not affect the subsequent frames.

  The streaming data is divided into a plurality of parts, the streaming data constructed at the minimum required bit rate is set as the main streaming data, and other sub-streaming data is added to the main streaming data to construct the higher bit rate streaming data. By adopting the streaming data structure, multi-bit rate streaming is realized without increasing the processing load of the streaming data creation server. Even if the communication speed temporarily decreases due to the influence of the radio wave condition of the mobile phone, etc. However, there is an advantage that streaming broadcast reception can be continued stably without interruption.

  It is possible to divide the screen of the mobile phone terminal for streaming viewing into two parts, stream live video with audio on one side, and display text information or still images on the other side. Can be used, such as displaying advertisements and advertisements, and can be widely applied commercially.

  Since the streaming data creation server can separately capture audio, video and text information, for example, a baseball TV video shot from multiple angles, radio broadcast audio explaining the same game, Stream and distribute the text information of the baseball player's performance data updated in real time on the Internet to mobile phone terminals, or pre-stream DVD contents with multi-angle and multi-language audio and subtitles It can be used as a video-on-demand service and distributed to mobile phone terminals as a video on-demand service.

  By distributing the processing to create streaming data by compressing and converting live audio, video, and text information, and distributing the streaming data to different servers, both processes can be performed stably in real time. In addition, there is an excellent advantage that the number of servers can be easily increased or decreased even when the scale of streaming distribution is expanded or reduced in the future.

Hereinafter, an example of a system and an example of a method corresponding to the system will be described with reference to the drawings, and an example of a program and an example of a recording medium for realizing the example of the method will be described. However, this invention is not limited to embodiment described below, A various change is possible within the range of the technical thought.
As an example, a case will be described in which live audio, video, and text information being shot with a television camera is streamed in real time on the NTT Docomo mobile phone terminal model “FOMA901i series”.

(System example)
First, FIG. 1 is a block diagram showing a configuration example of the entire system according to an embodiment of the present invention. As shown in the figure, this system includes a streaming data creation server 10, an Internet communication network 20, an Internet broadcast server 30, and a mobile phone terminal 40.

The streaming data creation server 10 captures live audio, video, and text information as a source, and is connected to the Internet communication network 20 to create streaming data that stores voice, video, and text data, and simultaneously stream the streaming data. Data is immediately distributed to the Internet broadcast server 30 via the Internet communication network 20.
Note that the number of streaming data creation servers 10 is not limited to one as shown in FIG. 1, and there may be a plurality of streaming data creation servers 10 connected to the Internet communication network 20.

  A streaming data creation server 10, an Internet broadcast server 30, and a mobile phone terminal 40 are connected to the Internet communication network 20. However, a mobile phone base station, a telephone line, a telephone exchange, and the like (not shown) are relayed between the Internet communication network and the mobile phone terminal.

The Internet broadcast server 30 receives, on the one hand, streaming data distributed from the streaming data creation server 10 via the Internet communication network 20 and simultaneously stores and accumulates it in its own storage device, and on the other hand, also via the Internet communication network 20. The mobile phone terminal 40 receives a request for distributing various data, and distributes data that meets the request to the mobile phone terminal 40.
The number of Internet broadcast servers 30 is not limited to one as shown in FIG. 1, and there may be a plurality of Internet broadcast servers 30 connected to the Internet communication network 20.

  Of various data distributed to the mobile phone terminal 40 by the Internet broadcast server 30 via the Internet communication network 20, the streaming data is limited to live audio, video, and text information distributed from the streaming data creation server 10. Instead, it may be a DVD content or the like that is converted into streaming data in advance and held in its storage area by the Internet broadcast server. In this case, a video-on-demand DVD broadcasting system and method for mobile phones is constructed.

The mobile phone terminal 40 is connected to the Internet communication network 20 via a mobile phone base station, a telephone line, a telephone exchange, etc. (not shown), and streaming content is transmitted to the Internet broadcast server 30 via the Internet communication network 20. And the streaming data corresponding to the request distributed from the Internet broadcast server 30 via the Internet communication network 20 is received and stored and stored in its storage device at the same time. Audio, video, and text information are restored from the above, and audio, video, and text information are reproduced in time, thereby realizing real-time streaming viewing of live broadcasts.
The number of mobile phone terminals 40 is not limited to one as shown in FIG. 1, and mobile phone terminals 40 connected to the Internet communication network 20 via a mobile phone base station, a telephone line, a telephone exchange, etc. (not shown) There may be multiple.

In the present embodiment, a configuration example of streaming data for realizing real-time streaming viewing of a live broadcast on the mobile phone terminal 40 will be described with reference to FIG. As shown in FIG. 2, the streaming data in this embodiment stores audio data, character information data, and still image data.
As the audio data, an ADPCM format having a length of 15 seconds, a quantization bit number of 16 bits, and a sampling frequency of 8 kilohertz, which is a part of a so-called “ringtone” that can be reproduced by a function unique to the mobile phone terminal 40 is used.
The still image data is independent from each other, and the size is 240 × 180 pixels suitable for the screen of the mobile phone terminal. These still images are sequentially displayed on the screen of the mobile phone terminal at a rate of 3 frames per second to reproduce the video.
Note that the audio data and the still image data are not limited to the above format, and may be in other formats.
In this embodiment, since the audio data and the still image data have different formats suitable for each model of the mobile phone terminal, the streaming data creation server uses the same live voice, live video, and character information for each model of the mobile phone terminal. Create streaming data in a format suitable for each.

As shown in FIG. 2, in this embodiment, multi-bit rate streaming is realized by dividing streaming data into two parts, main streaming data and sub-streaming data.
The main streaming data includes management information for indicating the contents of the data included in the main streaming data, audio data for 15 seconds, character information data, and still image data for the first frame for 15 seconds, a total of 15 frames. , Is stored.
On the other hand, the sub-streaming data stores management information included in the sub-streaming data, and still image data of the second and third frames per second for 15 seconds, a total of 30 pieces.

The mobile phone terminal 40 receives both the main streaming data and the sub-streaming data, so that the voice data, the character information data, and the still image data of 3 frames per second for a total of 15 seconds, a total of 45 frames, Get. However, when the communication speed is reduced due to the deterioration of the radio wave condition or the like, even if only the main streaming data is received, the video and text information of 1 frame per second synchronized with the voice for 15 seconds can be streamed.
Note that the number and method of dividing the streaming data for converting the streaming data into multi-bit rate streaming are not limited to the above conditions, and may be different.

In addition, subtitle data storing management information and character information is used to display subtitles and the like of a DVD broadcast in the 240 × 60 pixel area at the bottom of the screen of the mobile phone terminal during streaming viewing. Similarly, advertisement data storing management information, character information, and still images is used to display advertisements such as character information and still images at the bottom of the screen of the mobile phone terminal during streaming viewing.
Note that the area for displaying subtitles or advertisements is not limited to the above-mentioned size, and may be other values.

  Since the subtitle data and the advertisement data have a relatively small amount of data, and the contents can be prepared in advance without requiring real-time properties, real-time streaming viewing of the main streaming data and sub-streaming data is often performed. It is desirable that the mobile phone terminal receives and stores the caption data and the advertisement data necessary before starting the streaming viewing so as not to disturb the above.

Next, a configuration example of the streaming data creation server 10 will be described.
As shown in FIG. 3, the streaming data creation server 10 in this embodiment includes a CPU 11, a ROM 12, a RAM 13, an HDD 14, a streaming data creation unit 15, a communication unit 160, a streaming data distribution unit 161, a video input unit 170, and an audio input unit. 171, a character information input unit 172, a live source capturing unit 173, and a bus 18 that interconnects them.
Various control programs and the like are stored in the ROM 12, and the CPU 11 reads these control programs and performs various control processes. At this time, the RAM 13 is used as a work area for the CPU 11.

  The HDD 14 internally stores the streaming data creation / delivery program 141 for controlling each part of the streaming data creation server to create the streaming data and deliver it to the Internet broadcast server 30 and the streaming data stored until the created streaming data is delivered. A buffer 142 is stored.

The video input unit 170 includes a video shooting device such as a TV camera and a video data digitizing device such as a video capture board. The video data digitizing device converts the live video shot by the video shooting device in real time by the video data digitizing device. And is delivered to the streaming data creation / delivery program 141 via the live source capturing unit 173.
The video input unit 170 is not limited to a combination of a video shooting device and a video data digitizing device, but may be a TV capture unit that receives a TV program from an antenna and digitally outputs it in real time.

The voice input unit 171 includes a voice acquisition device such as a microphone and a voice data digitizing device such as a voice input sound board. The voice input unit 171 converts the live voice acquired by the voice acquisition device in real time by the voice data digitizing device and converts the digital voice. Data is created and passed to the streaming data creation / delivery program 141 via the live source capturing unit 173.
Note that the audio input unit 171 is not limited to a combination of an audio acquisition device and an audio data digitizing device, but may be a TV capture unit that receives a radio program from an antenna and digitally outputs it in real time.

In order to store the content of subtitles displayed in synchronization with live audio and video in the streaming data, the character information input unit 172 streams the subtitle content, display start time, and display end time in advance in this embodiment. The data is input to the data creation server 10, and these data are transferred to the streaming data creation / delivery program 141 via the live source capturing unit 173.
In addition, the character information input unit is not limited to the above-described configuration, and may be a unit that captures character information from the outside in real time via the Internet communication network and the communication unit and passes it to the streaming data creation / delivery program 141.

During live broadcasting, live video data, live audio data, and character information data are always captured by the live source capturing unit 173, and the streaming data creation / delivery program 141 in the HDD 14 is executed by the CPU 11. At this time, the RAM 13 is used as a work area for the CPU 11.
The communication unit 160 is connected to the Internet communication network 20 and immediately distributes the streaming data created by the streaming data creation / delivery program 141 to the Internet broadcast server 30 via the Internet communication network 20.

  Note that the number of streaming data creation servers 10 is not limited to one, but may be a plurality of streaming data creation servers 10 that can communicate with the Internet broadcast server 30 via the Internet communication network 20, and thus a plurality of streaming data creation servers 10 may be used. When capturing the source, the processing load per one streaming data creation server 10 can be easily distributed.

Next, a configuration example of the Internet broadcast server 30 will be described.
As shown in FIG. 4, the Internet broadcast server 30 in this embodiment includes a CPU 31, a ROM 32, a RAM 33, an HDD 34, a communication unit 350, a streaming data storage / storage unit 351, a menu management data distribution request receiving unit 352, and a menu management data distribution unit. 353, a streaming content distribution request receiving unit 354, a streaming content distribution unit 355, a streaming data management unit 36, a streaming content distribution management unit 37, and a bus 38 interconnecting them.
Various control programs and the like are stored in the ROM 32, and the CPU 31 reads these control programs and performs various control processes. At this time, the RAM 33 is used as a work area for the CPU 31.

  The communication unit 350 is connected to the Internet communication network 20 and communicates with the streaming data creation server 10 and the mobile phone terminal 40.

  The HDD 34 relates to an Internet broadcast program 341 for controlling each part of the Internet broadcast server 30 to distribute streaming content to the mobile phone terminal 40, a streaming buffer 342 for storing and storing streaming data, and a distributable streaming content. Menu management data 343 indicating information and distribution management data 344 for managing the streaming distribution state of each mobile phone terminal 40 are stored.

  The Internet broadcast program 341 in the HDD 34 is always executed by the CPU 31, receives streaming data distributed from the streaming data creation server 10 from the communication unit 350 connected to the Internet communication network 20, and receives a streaming data storage / storage unit 351. In step S1, the data is stored and accumulated in the streaming buffer 342.

  The streaming buffer 342 is a storage area for storing streaming data received from the streaming data creation server 10 in the streaming data storage / storage unit 351 by the Internet broadcast program 341.

  In addition to streaming data, the streaming buffer 342 stores character information data and still image data used as subtitles and advertisements that are distributed to the mobile phone terminal 40 and displayed on the screen of the mobile phone terminal that is viewing the streaming content. is doing.

  On the other hand, the Internet broadcast program 341 communicates with the mobile phone terminal 40 via the communication unit 34 and the Internet communication network 20, and receives a menu management data request from the mobile phone terminal 40 at the menu management data distribution request receiving unit 352. The requested menu management data 343 is distributed to the mobile phone terminal.

  The menu management data 343 stores contents displayed on the menu screen of the mobile phone terminal 40, such as a list of data that the Internet broadcast server 30 can deliver from the streaming data delivery unit 355 to the mobile phone terminal 40. Each time the menu screen data is requested from the mobile phone 40, it is updated by the Internet broadcast program 341.

  Similarly, the Internet broadcast program 341 communicates with the mobile phone terminal 40 via the communication unit 34 and the Internet communication network 20, and the streaming content delivery request receiving unit 354 receives a streaming content delivery start request from the mobile phone terminal. The streaming data related to the requested streaming content is distributed from the streaming data distribution unit 355 to the mobile phone terminal 40.

  In the distribution management data 344, which mobile phone terminal 40 distributes which streaming content is distributed by the Internet broadcast server 30 via the streaming data distribution unit 355 is managed by the Internet broadcast program 341 in accordance with the change in the distribution state. The streaming data distribution management unit 37 appropriately updates the data.

Next, a configuration example of the mobile phone terminal will be described.
As shown in FIG. 5, the mobile phone terminal in this embodiment includes a CPU 41, ROM 42, RAM 43, communication unit 440, menu management data distribution request unit 441, menu management data reception unit 442, streaming content distribution request unit 443, streaming data. The storage / accumulation unit 444, the streaming data restoration unit 45, the audio output unit 460, the video output unit 461, the streaming viewing unit 462, the interface 47 unit, and the bus 48 for interconnecting them.
Various control programs and the like are stored in the ROM 42, and the CPU 41 reads these control programs and performs various control processes. At this time, the RAM 43 is used as a work area for the CPU 41.

  The RAM 43 internally requests the Internet broadcast server 30 to distribute various data, receives various data distributed from the Internet broadcast server 30, stores and stores the data one by one, and reproduces it. A streaming buffer 432 for storing and accumulating various data distributed from the Internet broadcast server 30 is stored.

  The interface unit 47 is composed of various buttons of the mobile phone terminal 40 and its control device, and receives button operations by the user and transmits them to the CPU 41.

  The communication unit 440 includes an antenna, a tuning circuit, and the like, and is connected to the Internet communication network 20 via a mobile phone base station, a telephone line, a telephone exchange, etc. (not shown), communicates with the Internet broadcast server 30, and distributes menu management data The request unit 441 requests the Internet broadcast server 30 to distribute menu management data, and the menu management data reception unit 442 receives the requested menu management data from the Internet broadcast server 30, and receives a streaming content distribution request unit. In 443, the Internet broadcast server is requested to start streaming content distribution, and the streaming data storage and storage unit 444 receives the streaming data distributed from the Internet broadcast server one by one. Storing accumulated in reaming buffer 432.

  The streaming data restoration unit 45 restores the streaming data in the streaming buffer 432 to the original audio, video, and character information as appropriate by the Internet broadcast viewing program, and stores the data in the streaming buffer 432 again until the actual output timing.

  The streaming viewing unit 462 passes the restored audio, video, and character information stored in the streaming buffer 432 to the audio output unit and the video output unit in accordance with the timing.

The audio output unit 460 is composed of a speaker of a mobile phone terminal and a control device thereof, and outputs audio of content distributed in a streaming manner.
Note that audio output is not limited to a speaker, and may be output via headphones or the like via an external output terminal.

  The video output unit 461 is composed of a screen of a mobile phone terminal and its control device, and displays a menu screen, video of content distributed in a streaming manner, and the like.

(Example method)
A method example of the present embodiment example related to the system example will be described. FIG. 6 is a sequence diagram showing a system implementation method from when the streaming data is created by the streaming data creation server 10 until it is stored and accumulated by the Internet broadcast server 30.

  The streaming data creation server 10 always executes a streaming data creation and distribution program 141, and compresses the captured video, audio, and character information in real time and converts them into streaming data (Sa1).

Here, an audio compression processing method according to the present embodiment will be described with reference to FIG.
The audio (Sb101) captured by the streaming data creation server 10 is first divided into units of 15 seconds in synchronization with the video (Sb102), and converted to linear audio with a quantization bit number of 16 bits and a sampling frequency of 8 kilohertz. (Sb103), the volume is equalized by normalization processing (Sb104), converted to an ADPCM format that can be played as a so-called “ringtone” by the unique function of the mobile phone terminal 40 (Sb105), and the file header of the completed audio file is deleted Yes (Sb106) However, the various conditions relating to the compression processing are not limited, and various conditions suitable for each model of the mobile phone terminal 40 for streaming viewing may be used.

Next, a video compression processing method will be described with reference to FIG.
The video (Sb201) captured by the streaming data creation server 10 is first divided in units of 15 seconds in synchronization with the sound (Sb202), and one still image is extracted every 1/3 second (Sb203). The size of each still image is reduced to 240 × 180 pixels in accordance with the screen size of the mobile phone terminal 40 (Sb204), and JPEG compression is performed (Sb205). However, this JPEG compression is performed under the condition that the size of the compressed still image file is within 6 kilobytes. Finally, the unified file header is deleted from all the still image files (Sb206). However, the various conditions related to the compression processing are not limited, and various conditions suitable for each model of the mobile phone terminal 40 for streaming viewing may be used, and other image compression such as PNG compression instead of JPEG compression. A method may be used.

Next, a streaming data creation processing method will be described with reference to FIG.
For a specific 15 seconds, the main streaming data is created by combining the audio file created as described above and the first still image file of the still image files, a total of 15 files, character information and management information. (Sb107). Also, for the same 15 seconds, the second and third images per second, a total of 30 images, and the management information are combined in the still image file to create substreaming data (Sb207).
These streaming data are created under conditions suitable for all models of the mobile phone terminals 40 that the Internet live broadcast service of the present invention supports.

Here, the configuration of the main streaming data and the sub-streaming data will be described with reference to FIG.
As shown in FIG. 2, since the main streaming data stores a 15-second audio file and the first still image file per second, it is possible to reproduce a 1-frame video per second synchronized with the 15-second audio. it can. Also, since the second and third of the 3 frames per second still image are stored in the sub-streaming data, 3 frames per second synchronized with the 15-second sound is combined with the main streaming data described above. Videos can be played.

  In this way, by dividing the streaming data into two, it is usually possible to play back a high frame rate video using both the main streaming data and the sub-streaming data, and if the radio wave condition of the mobile phone terminal deteriorates, etc. When the speed decreases, the present invention realizes multi-bit rate streaming delivery by performing low frame rate video reproduction using only main streaming data.

Here, with reference to FIG. 6, the description will return to the implementation method example of the embodiment having the above-described configuration.
The streaming data creation server 10 compresses the captured audio, video, and text information in real time to create streaming data. At the same time, the streaming data is immediately sent to the Internet broadcast server 30 via the Internet communication network 20. Delivered (Sa2).

  The Internet broadcast server 30 receives the streaming data distributed from the streaming data creation server 10, stores and accumulates it (Sa3).

Here, the streaming data storage / accumulation unit 351 of the Internet broadcast server 30 checks the streaming buffer 342, and if the free space interrupts a certain value, the free space exceeding the certain value is deleted in order from the old streaming data. Is secured (Sa4).
Note that the value of the free capacity to be secured in the streaming buffer 342 of the Internet broadcast server 30 is set flexibly so that the Internet broadcast server 30 operates stably.

When the streaming data management unit 36 of the Internet broadcast server 30 receives or deletes the streaming data, the streaming data management unit 36 updates the menu management data 343 to match the current state as needed (Sa5).
The above operation is always performed regardless of whether the mobile phone terminal 40 is actually viewing the streaming content.

Here, the menu management data 343 of the Internet broadcast server 30 will be described.
The menu management data 343 manages the contents of the menu displayed on the screen of the mobile phone terminal 40. For example, a plurality of menus in a multi-level menu that the user moves by button operation, the contents are charged. A content type indicating whether it is free, live broadcast or DVD broadcast, a content title, and the like are stored.
Note that the menu management data 343 is not necessarily a single file, and may be managed in a distributed manner with a plurality of files.

  Next, with reference to FIG. 8, an example of an implementation method when the mobile phone terminal 40 is viewing streaming content distributed from the Internet broadcast server 30 via the Internet communication network 20 in the present embodiment having the above configuration. explain.

First, the Internet broadcast viewing program 431 is activated on the mobile phone terminal 40 (Sc1).
In the present embodiment, the Internet broadcast viewing program 431 is created and executed as an i-αppli of NTT DoCoMo, but is not limited thereto, and may be created and executed by a method suitable for the mobile phone model.

  When the Internet broadcast viewing program 431 is activated, first, the menu management data distribution request unit 441 requests the menu management data 343 from the Internet broadcast server 30 (Sc2). At this time, information on the model of the mobile phone terminal 40 itself is also transmitted.

  The Internet broadcast server updates the menu management data 343 (Sc3), and transmits the menu management data 343 including a list of streaming contents that can be viewed on the model of the mobile phone terminal 40 to the mobile phone terminal 40 ( Sc4).

  When receiving the menu management data 343 from the Internet broadcast server 30, the Internet broadcast viewing program 431 of the mobile phone terminal 40 displays a menu screen (Sc5).

  The user operates the menu screen to select and determine the streaming content to be viewed (Sc5), and transmits a streaming distribution start request for the selected content to the Internet broadcast server (Sc6).

  Here, a method for implementing a menu screen of the Internet broadcast viewing program 431 in the mobile phone terminal 40 will be described with reference to FIG.

  The menu screen of the Internet broadcast viewing program 431 is configured by a multi-level menu selection method, and the user selects an option displayed on the screen by pressing the upper button or the lower button. Next, the selected option is determined by pressing a determination button, or the previous screen is returned by pressing a return button.

In the present embodiment, the “Notification” page for the purpose of introducing new content is displayed first as the menu screen of the Internet broadcast viewing program.
Sd4 in FIG. 9 is an example of the “notification page”, but when the return button is pressed on the “notification” page, the “main menu” page is displayed.

  Sd1 in FIG. 9 is an example of a “main menu” page. The user follows a multi-level menu from here to select a live broadcast or a DVD broadcast and decides viewing.

  In addition, as with Sd6 or Sd7 in FIG. 9, for paid content, charging may be confirmed on the screen to the user.

Here, the description returns to FIG.
When receiving the streaming content distribution request from the mobile phone terminal 40, the Internet broadcast server 30 updates the distribution management data 344 (Sc7), and sequentially sends the streaming data of the requested streaming content to the mobile phone terminal 40. And continuously distributed (Sc8).

  The mobile phone terminal 40 sequentially stores streaming data continuously distributed from the Internet broadcast server 30 in the streaming buffer 432 in order (Sc9), and then stores still image data, audio data, and audio data stored in the streaming data to be reproduced next. The character information is restored (Sc10), and the next reproduction is started at the same time as the previous reproduction is finished (Sc11), and the continuous reproduction is performed to realize a long-time streaming viewing without a joint.

Here, the distribution management data 344 of the Internet broadcast server will be described.
The distribution management data 344 manages data related to streaming content distribution to the mobile phone terminal 40, and includes a user identification number of the mobile phone terminal 40, an identification number of viewing content, a viewing date, a viewing fee system, Etc. are stored.
The contents of the distribution management data are not limited to the above data, and other necessary data may be stored. Further, the distribution management data 354 is not necessarily a single file, and may be distributed and managed by a plurality of files.

Here, a streaming data restoration method in the mobile phone terminal 40 will be described with reference to FIG.
When the mobile phone terminal 40 receives the streaming data distributed from the Internet broadcast server 30, it temporarily stores and stores it in the streaming buffer 432 (Se101), and receives the next streaming data (Se102), and repeats.

Of the streaming data stored in the streaming buffer 432 of the mobile phone terminal 40, the data to be reproduced next is restored to the original audio, still image, and character information data.
The streaming data is simply binary-combined with audio data or still image data, so it is divided according to the contents of the management part.

  The audio data is extracted from the main streaming data (Se202), a file header for “ringtone” is added (Se303), and stored in the streaming buffer 432 of the mobile phone terminal 40 in the “ringtone” file format (Se304). .

  At least 15 still image data from the main streaming data, and 30 sub-stream data if stored and accumulated in the streaming buffer 432, a maximum of 45 images are extracted (Se202). A file header is added (Se403), and is stored and accumulated in the streaming buffer 432 in a still image file format (Se404).

  The streaming viewing unit 462 of the mobile phone terminal 40 monitors the timing of streaming playback (Se205), and starts the next playback at the same time as the playback of the currently played audio, still image, and character information is completed, thereby interrupting the playback. Realize long playback.

  The audio data is reproduced by using a function of playing a so-called “ringtone” inherent to the mobile phone terminal 40, but if two “ringtones” are reproduced continuously, the two are ignored between the two “ringtones”. Since it becomes difficult to hear because there is no silent time, using the two audio playback channels alternately starts playback of the next audio simultaneously with the end of the previous audio playback, that is, simultaneously with the start of the silence time. Thus, it is possible to reproduce a sound file for a long time without interruption (Se306).

The first still image is displayed simultaneously with the start of audio reproduction (Se406), and then the next still image is displayed every unit time, thereby realizing the reproduction of moving images in a pseudo manner (Se407).
Here, the unit time for switching the display of still images is 1/3 second if the number of still images is 45, and 1 second if 15 still images, and is synchronized with the sound for 15 seconds.

  Here, referring to FIG. 11, in the present embodiment in the above configuration, there is less room for streaming data accumulated in the mobile phone terminal 40 during streaming delivery, and switching to streaming viewing at a low bit rate is performed. A method example will be described.

  When the margin of streaming data stored in the mobile phone terminal 40 decreases due to a decrease in the communication speed of the mobile phone terminal 40 due to a change in the radio wave condition of the mobile phone terminal 40 (Sf1), the mobile phone terminal 40 The Internet broadcast viewing program 431 requests the Internet broadcast server 30 to distribute only the main streaming data (Sf2), receives only the main streaming data from the Internet broadcast server 30 (Sf3), and uses only the main streaming data. Switch to low bit rate streaming viewing.

In the present invention, in order to ensure the continuity of streaming viewing, the mobile phone terminal 40 first secures at least 30 seconds of main streaming data within the streaming buffer 432 with a margin.
However, in the case of live broadcasting, there is a case where there is no streaming data to be streamed next, because streaming playback on the mobile phone terminal 40 catches up with streaming data creation on the streaming data creation server 10, and so on. The main streaming data to be secured may be for 15 seconds.

  In low bit rate streaming distribution, the video that can be displayed by the mobile phone terminal 40 is 1 frame per second, but the audio is the same as the high bit rate streaming distribution and the communication speed is about half of the normal speed. Even if the communication speed of 40 decreases, stable streaming delivery can be continued (Sf4). Meanwhile, the streaming data is accumulated in the streaming buffer 432 of the mobile phone terminal 40 with a sufficient margin.

  After that, when a sufficient margin is obtained for the streaming data stored in the streaming buffer 432 of the mobile phone terminal 40 (Sf5), the Internet broadcast viewing program 431 of the mobile phone terminal 40 sends the Internet broadcast server 30 to the Internet broadcast server 30. A distribution request for streaming data including sub-streaming data in addition to the main streaming data is transmitted (Sf6).

  The Internet broadcast server 30 continuously distributes high bit rate streaming data, that is, both main streaming data and sub-streaming data in accordance with the request from the mobile phone terminal 40 (Sf7). Return to the bit rate streaming viewing (Sf8).

  Here, with reference to FIG. 12, an example of an implementation method in the case where an overflow occurs in the streaming buffer 432 of the mobile phone terminal 40 during streaming distribution in the present embodiment having the above configuration will be described.

  The mobile phone terminal 40 continues to receive streaming data from the Internet broadcast server 30 and store and store it in the streaming buffer 432 in order to prepare for an unexpected situation such as a temporary interruption of communication due to a change in radio wave conditions. If there is no room in the streaming buffer 432 of the mobile phone terminal 40, the streaming buffer 432 overflows (Sg1), and no more streaming data can be received, so the streaming distribution request to the Internet broadcast server is suspended. (Sg2).

  Since the Internet broadcast server 30 does not distribute the next streaming data to the mobile phone terminal 40 until the streaming delivery request from the mobile phone terminal 40 is received, the mobile phone terminal 40 is stored in the streaming buffer 432. Streaming viewing is continued using only the streaming data (Sg3).

  Thereafter, streaming viewing of the mobile phone terminal 40 progresses, the streaming data is digested, a free space in the streaming buffer 432 is created, and the overflow of the streaming buffer 432 is resolved (Sg4), the mobile phone terminal 40 is connected to the Internet. The streaming distribution request to the broadcast server 30 is resumed (Sg5).

  When the Internet broadcast server 30 receives a streaming distribution reproduction request from the mobile phone terminal 40, the streaming data distribution to the mobile phone terminal 40 is resumed (Sg6), and the mobile phone 40 returns to normal streaming viewing. (Sg7).

  Here, with reference to FIG. 13, an example of an implementation method in the case where streaming data is insufficient in the streaming buffer 432 of the mobile phone terminal 40 during streaming viewing in the present embodiment having the above configuration will be described.

  The streaming distribution from the streaming data reproduction server 10 to the Internet broadcast server 30 is delayed, the streaming distribution from the Internet broadcast server 30 to the mobile phone terminal 40 is too advanced, or the mobile phone terminal 40 falls into communication failure. Even when streaming distribution from the broadcast server 30 to the mobile phone terminal 40 is stopped, streaming viewing on the mobile phone terminal 40 is continued as long as streaming data is accumulated in the streaming buffer 432 of the mobile phone terminal 40. When the streaming data stored in the streaming buffer 432 becomes small (Sh1), the mobile phone terminal 40 sends a streaming distribution request to the Internet broadcast server 30 and the streaming distribution is resumed. Regularly any number of times to send until the (Sh2).

  When streaming viewing of the mobile phone terminal 40 proceeds without streaming distribution from the Internet broadcast server 30 and all the streaming data is consumed, the streaming viewing is temporarily stopped (Sh3).

  When streaming distribution from the Internet broadcast server 30 to the mobile phone terminal 40 is restored (Sh4), streaming viewing of the mobile phone terminal 40 returns to normal (Sh5).

(Program example and recording medium example)
Finally, a streaming data creation server 10 that has the basic functions of a computer, takes in audio, video, and text information that is a source of Internet broadcasting, performs compression processing, and immediately delivers it as streaming data for mobile phones, The distributed streaming data is temporarily stored and accumulated and then distributed from the Internet broadcast server 30 and the Internet broadcast server 30 that distribute the streaming data to the mobile phone terminal 40 in accordance with the streaming data distribution request received from the mobile phone terminal 40. A description will be given of an example of a program that is executed by receiving the streaming data, restoring the audio, video, and text information from the streaming data and introducing the streaming data to the mobile phone terminal 40 for streaming viewing.

  The streaming data creation / distribution program 141 compresses and converts the processing procedure for capturing live audio, video, and character information, which is the source of streaming content, into the streaming data creation server 10, and the captured audio, video, and character information. Then, a processing procedure for creating streaming data suitable for each model of the mobile phone terminal and a processing procedure for immediately distributing the created streaming data to the Internet broadcast server via the Internet communication network 20 are sequentially executed.

The Internet broadcast program 341, on the one hand, receives the streaming data distributed from the streaming data creation server 10 via the Internet communication network 20 one by one, and stores and stores it in the streaming buffer 342 one by one. A process procedure for updating the menu management data 343 in response to a change in the content of the streaming buffer 342,
On the other hand, the processing procedure for receiving the distribution request for the menu management data 343 transmitted from the mobile phone terminal 40 via the Internet communication network 20 and the menu management data 343 requested from the mobile phone terminal 40 are A processing procedure for transmitting to the mobile phone terminal 40, a processing procedure for receiving various requests relating to distribution of streaming data transmitted from the mobile phone terminal 40, and updating the distribution management data 344 in response to the request, and the Internet The processing procedure for delivering streaming data to the mobile phone terminal 40 via the communication network 20 is repeatedly executed.

The Internet broadcast viewing program 431 first requests the Internet broadcast server 30 to distribute the menu management data necessary for displaying the menu screen to the mobile phone terminal 40 via the Internet communication network 20. A processing procedure for receiving the requested menu management data, a processing procedure for displaying a menu screen based on the received menu management data, and selecting streaming content to be viewed by operating the menu screen, and an Internet communication network A processing procedure for requesting the Internet broadcast server 30 to start distribution of the selected streaming content via the Internet 20 in sequence,
Also, a processing procedure for receiving streaming data distributed from the Internet broadcast server 30 via the Internet communication network 20, a processing procedure for storing and storing the received streaming data in the streaming buffer 342, audio from the stored streaming data, A processing procedure for restoring video and text information and a processing procedure for synchronously reproducing the restored audio, video and text information are sequentially executed repeatedly,
Further, when there is less room for streaming data stored in the mobile phone terminal 40, a processing procedure for requesting distribution of only the main streaming data to the Internet broadcast server and only the main streaming data from the Internet broadcast server are received. A processing procedure, a processing procedure for performing low bit rate streaming viewing, and a processing procedure for returning to high bit rate streaming viewing when there is room for accumulation of streaming data are appropriately executed,
In addition, when an overflow occurs in the streaming buffer 432 of the mobile phone terminal 40, streaming viewing using only the streaming data stored in the streaming buffer 432 and the processing procedure for temporarily stopping the streaming data distribution request to the Internet broadcast server. And a processing procedure for restarting the streaming delivery request to the Internet broadcast server and returning to the normal streaming viewing when the overflow of the streaming buffer 432 is resolved,
Further, when the shortage of streaming data in the streaming buffer of the mobile phone terminal 40 occurs, the Internet broadcast server is requested to resume the streaming data distribution via the Internet communication network 20 until the streaming data distribution is resumed. The processing procedure is repeatedly executed periodically.

  At this time, the respective recording media for actually recording the above three program examples including a series of processing procedures as computer-readable practical procedures are obtained.

  As described above, the embodiment of the present invention has been described by taking as an example the case where live audio, video, and character information is captured, compressed and processed, streamed, and viewed on the mobile phone terminal 40. Is not necessarily limited to the above-described means, techniques, procedures, and procedures, and can be changed as appropriate within the scope of the above-described effects. For example, the entire streaming data is created in advance. The present invention is also applied to a case where a completed DVD is viewed on the mobile phone terminal 40 by the video on demand method.

It is a block diagram which shows the structure of the whole system in embodiment of this invention. It is a schematic diagram which shows the structure of the streaming data in the embodiment. It is a block diagram which shows the structure of the streaming data creation server in the same embodiment. It is a block diagram which shows the structure of the internet broadcast server in the embodiment. It is a block diagram which shows the structure of the mobile telephone terminal in the same embodiment. It is a sequence diagram which shows operation | movement until a streaming data creation server produces | generates streaming data and the internet broadcast server memorize | stores this in the same embodiment. In the same embodiment, it is a flowchart figure in which a streaming data production server produces streaming data. FIG. 6 is a sequence diagram showing an operation from when the mobile phone terminal starts an Internet broadcast viewing program until the streaming content is viewed in the embodiment. In the embodiment, it is a schematic diagram which shows an example of the menu screen of the internet broadcast viewing program displayed on the screen of a mobile phone terminal. FIG. 5 is a flowchart until the mobile phone terminal receives streaming data and views the restored audio and video synchronously in the embodiment. FIG. 6 is a sequence diagram illustrating an operation from when the margin of streaming data stored in the mobile phone terminal is reduced until the recovery is performed in the embodiment. FIG. 6 is a sequence diagram showing an operation from the overflow of the streaming buffer of the mobile phone terminal to recovery in the embodiment. FIG. 6 is a sequence diagram showing an operation from when the remaining streaming data accumulated in the mobile phone terminal is reduced until it is recovered in the embodiment.

Explanation of symbols

10 Streaming data creation server 11 CPU
12 ROM
13 RAM
14 HDD
141 Streaming Data Creation / Distribution Program 142 Streaming Buffer 15 Streaming Data Creation Unit 160 Communication Unit 161 Streaming Data Distribution Unit 170 Video Input Unit 171 Audio Input Unit 172 Character Information Input Unit 173 Live Source Capture Unit 18 Bus 20 Internet Communication Network 30 Internet Broadcast Server 31 CPU
32 ROM
33 RAM
34 HDD
341 Internet broadcast program 342 Streaming buffer 343 Menu management data 353
344 Distribution management data 354
350 Communication Unit 351 Streaming Data Storage and Storage Unit 352 Menu Management Data Distribution Request Receiving Unit 353 Menu Management Data Distribution Unit 354 Streaming Content Distribution Request Receiving Unit 355 Streaming Data Distribution Unit 36 Streaming Data Management Unit 37 Streaming Content Distribution Management Unit 38 Bus 40 Mobile Telephone terminal 41 CPU
42 ROM
43 RAM
431 Internet broadcast viewing program 432 Streaming buffer 440 Communication unit 441 Menu management data distribution request unit 442 Menu management data reception unit 443 Streaming content distribution request unit 444 Streaming data storage accumulation unit 45 Streaming data restoration unit 460 Audio output unit 461 Video output unit 462 Streaming viewing part 47 Interface part 48 Bus

Claims (10)

Streaming data creation server that creates and distributes streaming data obtained by compressing and converting live audio, video and text information in real time; stores and stores the streaming data received via the Internet communication network; and distributes it from mobile phone terminals An Internet broadcast server for delivering the stored and accumulated streaming data in response to a request;
A mobile phone terminal capable of synchronously streaming and outputting audio, video, and text information restored from the streaming data received from the Internet broadcasting server via the Internet communication network in real time without interruption. An internet live broadcasting system for mobile phones,
The streaming data creation server includes:
Captures live audio acquired by the audio acquisition device, live video captured by the video imaging device, the live audio stored in advance in the streaming server, and character information that is the content of the subtitles displayed in synchronization with the live audio Live source capture means,
The captured live audio is compressed and processed as digital audio data so that it can be played back as a ringtone that is a function unique to a mobile phone terminal in synchronization with the video data in real time, and the captured live video is compressed in real time. In addition to extracting a still image by dividing into a plurality of data in synchronization with the data, the digital audio data and the digital video data are further processed by compressing the digital audio data and the digital video data as reproducible digital video data. Streaming data creation means for creating streaming data suitable for each mobile phone model to be combined with,
Streaming data distribution means for immediately distributing the created streaming data to the Internet broadcast server via the Internet communication network;
Comprising
The Internet broadcast server is
Streaming data storage and storage means for receiving streaming data that is constantly distributed from the streaming data creation server via the Internet communication network one by one, and storing and storing in a storage device;
Streaming data management means for managing the stored and accumulated streaming data;
Menu management data distribution request receiving means for receiving a menu management data distribution request from the mobile phone terminal via the Internet communication network;
Menu management data distribution means for updating the menu management data requested from the mobile phone terminal via the Internet communication network and distributing it to the mobile phone terminal;
A streaming content distribution request receiving means for receiving a streaming content distribution request from the mobile phone terminal via the Internet communication network;
Streaming data distribution means for distributing streaming data requested from the mobile phone terminal to the mobile phone terminal via the Internet communication network;
Streaming content distribution management means for managing distribution of streaming content to the mobile phone terminal;
Comprising
The streaming data is
Video data and audio data, each of which is divided for each unit time and each data size is a size that can be accommodated in the storage area in the mobile phone terminal , and is composed of independent still image data Including
The mobile phone terminal
Menu management data distribution request means for requesting menu management data necessary for displaying a menu screen to the Internet broadcast server via the Internet communication network;
Menu management data receiving means for receiving the requested menu management data from the Internet broadcast server via the Internet communication network;
Streaming content distribution request for selecting streaming content from a menu screen displayed based on the received menu management data and requesting the Internet broadcast server to start distributing the selected streaming content via the Internet communication network Means,
Streaming data storage and storage means for storing and storing streaming data continuously distributed from the Internet broadcast server in the storage device one by one via the Internet communication network;
Streaming data restoring means for restoring various data such as live audio, video and text information from the stored and accumulated streaming data;
Streaming viewing means having an audio output unit that reproduces live audio in real time and a video output unit that reproduces video and text information synchronously with the audio, using the restored various data,
Comprising
The audio output unit
Having a plurality of audio reproduction channels used alternately so that continuous audio data can be reproduced without interruption for a long time,
A live internet broadcasting system for mobile phones. The streaming data creation means of the streaming data creation server includes:
The captured live audio, video, and text information is synchronously divided every unit time,
The separated live audio is compressed and converted into an audio file in a format that can be output by an audio output function that is an essential function of the mobile phone terminal,
On the other hand, a still image is extracted every unit time from the divided live video,
Each of the extracted still images is compressed and converted into independent still image files in a format that can be displayed by the still image display function that the mobile phone terminal has as a basic function,
Stores management information indicating information such as the compression-converted audio file, a series of still-image files that have been compression-converted, selected at regular intervals, the character information file, and the size of these files, and the like. Main streaming data that can be streamed at a low frame rate at a single rate,
The management information indicating the rest of the compressed series of still image files and the size of these files is stored, and streaming viewing with a high bit rate and a high frame rate is possible only when combined with the data stored in the main streaming data. Substreaming data configured to be possible,
Is configured to create multi-bit rate streaming data for each mobile phone model,
The streaming data distribution means of the Internet broadcast server includes:
Normally, high bit rate streaming delivery is performed by delivering main streaming data and sub streaming data, which are all of the multi-bit rate streaming data, to the mobile phone terminal,
When the communication speed is insufficient to perform the high bit rate streaming distribution, the low bit rate streaming is performed by distributing only the main streaming data which is a part of the multi-bit rate streaming data to the mobile phone terminal. Configured to deliver,
The streaming viewing means of the mobile phone terminal is:
When outputting a video by sequentially displaying a series of still image files obtained by the streaming data restoration means every unit time, the series of still images is displayed according to the bit rate at the time of reception. By dynamically changing and setting the frame rate to be displayed, it is configured to play back and output audio and video by multi-bit rate streaming data synchronously.
The Internet live broadcast system for mobile phones according to claim 1. The streaming viewing means of the mobile phone terminal is:
The screen display of the mobile phone terminal is divided into two parts, and one is configured to play and stream live video in real time, and the other is configured to play and display subtitles and advertisements as text information or still images.
The Internet live broadcast system for mobile phones according to claim 1 or 2. The streaming data storage and storage means of the Internet broadcast server includes:
It is configured to store and accumulate streaming data of existing content that has already been compressed and converted.
The Internet live broadcast system for mobile phones according to claim 1, 2, or 3. Streaming data created in real time by the streaming data creation server from live audio, video and text information is distributed to mobile phone terminals via the Internet communication network and Internet broadcast server.
An internet live broadcasting method for mobile phones that performs streaming playback output of the live audio, video and text information synchronously in real time without interruption for a long time on the mobile phone terminal,
The streaming data creation server includes:
First, live audio acquired by an audio acquisition device, live video captured by a video imaging device, live audio stored in a streaming server in advance, and subtitles displayed in synchronization with the live audio by live source capture processing The character information that becomes the contents of
Next, the captured live video is compressed and processed as digital audio data so that it can be reproduced as a ringtone that is unique to a mobile phone terminal in synchronization with the video data in real time, and the captured live video is processed in real time. In synchronization with the audio data, it is divided into a plurality of images and extracted as still images. The digital audio data and the digital video data are further compressed and processed as digital video data that can be played back by the mobile phone terminal. In combination with the character information, each data size can be stored in a storage area in the mobile phone terminal, and the video data is composed of still image data independent from each other. Create streaming data,
Subsequently, by streaming data distribution processing, the streaming data is immediately distributed to the Internet broadcast server via the Internet communication network,
On the other hand, the Internet broadcast server
Streaming data storage and accumulation processing, receiving the streaming data received from the streaming data creation server via the Internet communication network one by one, store and accumulate,
The streaming data management process manages the stored and accumulated streaming data,
A menu management data distribution request reception process receives a menu management data distribution request from the mobile phone terminal via the Internet communication network.
The menu management data distribution process updates the menu management data requested from the mobile phone terminal via the Internet communication network, and distributes it to the mobile phone terminal.
Streaming content distribution request reception processing receives a streaming content distribution request from the mobile phone terminal via the Internet communication network,
Through streaming data distribution processing, the streaming data requested from the mobile phone terminal is distributed to the mobile phone terminal,
Streaming content distribution management processing manages the distribution of streaming content to the mobile phone terminal,
On the other hand, the mobile phone terminal
Requesting menu management data necessary for displaying the menu screen to the Internet broadcast server via the Internet communication network by the menu management data distribution request process,
The menu management data receiving process receives the requested menu management data from the Internet broadcast server via the Internet communication network,
Streaming content delivery request processing selects streaming content from a menu screen displayed based on the received menu management data, and starts delivery of the selected streaming content to the Internet broadcast server via the Internet communication network Request and
Through streaming data storage and accumulation processing, streaming data distributed from the Internet broadcast server via the Internet communication network is stored and accumulated in the storage device one by one,
Various data such as live audio, video and text information are restored from the streaming data by the streaming data restoration process.
By streaming playback processing, the audio, video, and text information are continuously played back in time, and live audio, video, and text information can be synchronized synchronously for a long time using the restored data. Playback and output in real time,
A live internet broadcasting method for mobile phones. The streaming data creation process of the streaming data creation server includes:
The captured live audio, video, and text information is synchronously divided every unit time,
The separated live audio is compressed and converted into an audio file in a format that can be output by an audio output function that is an essential function of the mobile phone terminal,
On the other hand, a still image is extracted every unit time from the divided live video,
Each of the extracted still images is compressed and converted into independent still image files in a format that can be displayed by the still image display function that the mobile phone terminal has as a basic function,
Stores management information indicating information such as the compression-converted audio file, a series of still-image files that have been compression-converted, selected at regular intervals, the character information file, and the size of these files, and the like. Main streaming data that can be streamed at a low frame rate at a single rate,
The management information indicating the rest of the compressed series of still image files and the size of these files is stored, and streaming viewing with a high bit rate and a high frame rate is possible only when combined with the data stored in the main streaming data. Substreaming data configured to be possible,
To create multi-bit rate streaming data for each mobile phone model,
The streaming data distribution process of the Internet broadcast server is
When the communication speed of the mobile phone terminal is sufficiently high, high bit rate streaming distribution is performed by distributing main streaming data and sub-streaming data that are all of the multi-bit rate streaming data to the mobile phone terminal,
On the other hand, when the communication speed of the mobile phone terminal is low, low bit rate streaming delivery is performed by delivering only the main streaming data that is part of the multi-bit rate streaming data to the mobile phone terminal,
The streaming viewing process of the mobile phone terminal is as follows:
When outputting a video by sequentially displaying a series of still image files obtained by the streaming data restoration process every unit time, the series of still images is displayed according to the bit rate at the time of reception. By dynamically changing and setting the frame rate to be displayed, audio and video based on multi-bit rate streaming data can be reproduced and output synchronously.
The Internet live broadcasting method for mobile phones according to claim 5. The streaming viewing process of the mobile phone terminal is as follows:
Dividing the screen display of the mobile phone terminal into two parts, streaming playback output of live video in real time on one side, and display and output subtitles and advertisements as character information or still images on the other side,
The Internet live broadcasting method for mobile phones according to claim 5 or 6. The streaming data storage and accumulation process of the Internet broadcast server includes:
Store and store streaming data of existing content that has already been compressed and converted.
The Internet live broadcasting method for mobile phones according to claim 5, 6 or 7. A streaming data creation server that creates and distributes streaming data obtained by compressing and converting live audio, video, and character information in real time, as well as having the basic functions of a computer, and the streaming data received via the Internet communication network An Internet broadcast server that stores and stores the streaming data stored and stored in response to a distribution request from the mobile phone terminal, and a live data restored from the streaming data received from the Internet broadcast server via the Internet communication network. Streaming data applied to the streaming data creation server in a mobile phone Internet live broadcasting system constructed with a mobile phone terminal that reproduces and outputs audio, video and text information A formed distribution program,
In the streaming data creation server,
Live source import processing procedure to capture live audio, video and text information,
Streaming data creation processing procedure for creating the streaming data suitable for each mobile phone model by compressing and converting the captured live audio, video and text information in real time;
Streaming data distribution processing procedure for immediately distributing the created streaming data to the Internet broadcast server via the Internet communication network, and sequentially executing the streaming data,
The streaming data creation processing procedure includes:
A procedure for synchronously dividing the captured live audio, video and text information every unit time;
A procedure for compressing and converting the separated live audio into an audio file in a format that can be output by an audio output function that is an essential function of the mobile phone terminal;
On the other hand, a still image is extracted every unit time from the divided live video,
A procedure for compressing and converting each of the extracted still images into independent still image files in a format that can be displayed by the still image display function that the mobile phone terminal has as a basic function;
Stores the compression-converted audio file, the compression-converted series of still image files selected every certain number, the character information file, the total number of these files, and the management information indicating the information of each size. , Main streaming data that can be streamed at a low bit rate and low frame rate,
The management information indicating the rest of the compressed series of still image files and the size of these files is stored, and streaming viewing with a high bit rate and a high frame rate is possible only when combined with the data stored in the main streaming data. Substreaming data configured to be possible,
The multi-bit-rate streaming data is divided into, created for each model of the mobile telephone terminal,
The mobile phone terminal receives the streaming data from the streaming server, restores the audio data, the still image file, and the character information included in the streaming data, and has an audio output unit in the mobile phone terminal Continuous audio data is reproduced on a plurality of audio reproduction channels that are alternately used so as to be reproducible without interruption for a long time, and processing means for reproducing the still image file and the character information synchronously with the audio is sequentially executed. Let
A streaming data creation and distribution program characterized by the above. A procedure for executing each processing procedure in the streaming data creation and delivery program according to claim 9 is recorded in a computer-readable manner.
The recording medium which recorded the streaming data preparation delivery program characterized by the above-mentioned.