KR102540579B1 - Video conference server and video service providing method capable of audio synchronization in an online performance or video conference, and video service system using the same - Google Patents

Abstract

The present invention relates to a video conference server that provides audio-tunable video in an online performance or video conference, a method for providing the video service, and a video service system using the same. Selection that provides voice signals or audio signals of clients selected from among the plurality of clients participating in an online performance or video conference, connected to a network, and selected from among the clients in the online performance or video conference to the clients a multi-voice mode provider providing multiple voice support modes; a client IP selector for selecting a plurality of client IPs corresponding to the request of the first client when a first client among the plurality of clients requests the selective multi-voice support mode; and a client IP transmitter for transmitting the selected plurality of client IPs to the first client, wherein: When the first client receives the plurality of client IPs, the first client requests transmission of voice signals or audio signals from the plurality of clients corresponding to the plurality of client IPs, and when receiving voice signals or audio signals from the selected plurality of clients, the first client It can be output by mixing with video and audio signals of online performances or video conferences.

Description

A video conference server that provides audio-synchronizable video in an online performance or video conference, and a video service providing method thereof, and a video service system using the same, capable of audio synchronization in an online performance or video conference, and video service system using the same}

The present invention relates to a video system, and relates to an online performance or online performance that can smoothly provide a video service by selectively providing a preferred voice or audio signal even in a situation where real-time is important in a video conference or online performance accessed by an unspecified number of people. It relates to a video conferencing server that provides audio-tunable video in a video conference, a video service providing method thereof, and a video service system using the same.

1 shows the configuration of a general video conference system. Receives video and audio from all attendees (clients) participating in the video conference, mixes a plurality of received videos and audios to form a single stream, or displays a layout desired by each client. ) to create multiple streams and send them to each client. The video conferencing system shown in FIG. 1 collects the audio streaming channels of each participant (client) in the centralized video conferencing server 110, mixes them into one stream, and transmits the audio signal. A delay of about ~500 ms occurs. This is not a problem when the speaker and the listener speak in turn, such as in a video conference, but in the case of video services that are sensitive to delay, such as ensemble, group singing, and simultaneous reading, it is difficult to synchronize the voice due to the time difference of 400 to 500 ms.

Because of this problem, the online ensemble service uses a dedicated audio interface (reducing media encoding/decoding time) to reduce audio transmission delay, or uses the peer-to-peer method shown in FIG. shortening the time it takes) to provide services. For example, Yamaha sinkroom, jacktrip, etc. 2 shows the configuration of a P2P video conferencing system. Participants and participants are directly connected to each other to mix and transmit audio signals, thereby reducing the delay of real-time audio signals. However, when the number of participants is large, terminals and home Due to limitations in network performance, there are limitations in providing services to many participants.

Registered Patent Publication No. 10-2324609, 2021.11.04)

The problem to be solved by the present invention was created to solve the above-mentioned problems, and delay of voice or audio signal in real time by selectively providing preferred voice or audio signal in video conference or online performance accessed by an unspecified number of people. It is to provide a video conferencing server capable of synchronizing audio in an online performance or video conference and a method for providing the video service, which can provide video service smoothly even in situations where real-time is important, such as group singing, ensemble, and simultaneous reading of the Bible.

An object to be solved by the present invention is to provide a video service system using the video conferencing server capable of synchronizing audio in an online performance or video conference.

According to one aspect of the present invention for solving the above technical problem, a video conferencing server that provides audio-tunable video in an online performance or video conference participates in an online performance or video conference and is connected to a plurality of clients connected by a network. and a multi-voice mode providing unit providing a selective multi-voice support mode for providing voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients; a client IP selector for selecting a plurality of client IPs corresponding to the request of the first client when a first client among the plurality of clients requests the selective multi-voice support mode; and a client IP transmission unit configured to transmit the selected plurality of client IPs to the first client, wherein the first client receives the plurality of client IPs corresponding to the plurality of client IPs. When a voice signal or audio signal is received from the selected plurality of clients by requesting transmission of a voice signal or an audio signal, the video and audio signals of the online performance or video conference may be mixed and output.

According to an aspect of the present invention, a video conferencing server that provides audio-tunable video in an online performance or video conference includes at least one of a plurality of client IPs and a wired IP database and a wireless mobile IP database storing location information of the client IPs. The client IP selection unit may select the plurality of client IPs by obtaining location information of client IPs using at least one of the wired IP database and the wireless mobile IP database.

According to an aspect of the present invention, a video conferencing server that provides audio-tunable video in an online performance or video conference is a wired IP that stores the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. database and at least one of a wireless mobile IP database, wherein the client IP selector obtains location information and network operator information of the client IP using at least one of the wired IP database and the wireless mobile IP database, It is possible to provide an audio-tunable video in an online performance or video conference, characterized by selecting a client IP of.

When the client IP selection unit receives a request for the selective multi-voice support mode including at least one of preferred gender, age, and region information from the first client, the gender or gender information is determined through subscriber information or artificial intelligence (AI) voice analysis. Characterized in acquiring age information, it is possible to provide an audio-tunable video in an online performance or video conference.

According to another aspect of the present invention for solving the above technical problem, a video conferencing server providing audio tunable video in an online performance or video conference participates in an online performance or video conference and is connected to a plurality of clients connected by a network. and a multi-voice mode providing unit providing a selective multi-voice support mode for providing voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients; a client IP list generation unit generating a client IP list corresponding to the selective multi-voice support mode request of the first client when a first client among the plurality of clients requests the selective multi-voice support mode; and an IP list transmitter for transmitting the generated client IP list to the first client, wherein the first client requests the video conferencing server for the selective multi-voice support mode, and the video conferencing server Receives the client IP list from the client IP list, measures network delay for clients corresponding to the client IPs in the client IP list, requests voice signals or audio signal transmission to clients with less network delay, and requests voice signals Alternatively, when an audio signal is received, it may be output after being mixed with the video and audio signals of the online performance or video conference.

According to another aspect of the present invention, a video conferencing server providing audio-tunable video in an online performance or video conference includes a wired IP database and a wireless mobile IP database storing the plurality of client IPs and location information of the client IPs. The client IP list generating unit may generate the client IP list by obtaining location information of the client IP using at least one of the wired IP database and the wireless mobile IP database.

According to another aspect of the present invention, a video conferencing server that provides audio-tunable video in an online performance or video conference is a wired IP that stores the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. It may further include at least one of a database and a wireless mobile IP database, wherein the client IP list generator obtains location information of the client IP and network operator information using at least one of the wired IP database and the wireless mobile IP database. By doing so, the client IP list can be created.

When the client IP list generation unit receives a request for the selective multi-voice support mode including at least one of preferred gender, age, and region information from the first client, the client IP list generation unit uses subscriber information or artificial intelligence (AI) voice analysis to determine the gender. Alternatively, age information may be obtained.

According to another aspect of the present invention, a video conferencing server providing audio tunable video in an online performance or video conference is connected to a plurality of clients participating in an online performance or video conference to provide an audio tunable video service to the plurality of clients. A video conferencing server comprising: an additional information classification unit for acquiring the plurality of client IPs, storing and classifying them together with at least one of region, age, gender information, and network delay information; a client selection unit that selects and designates a plurality of clients with high speeds from among the plurality of clients using the classified information to reduce network delay; and a server mixing unit which receives voice signals or audio signals from the plurality of designated clients, mixes them with video and audio signals of the online performance or video conference, and transmits the signals to the plurality of clients.

According to another aspect of the present invention, the video conferencing server that provides audio-tunable video in an online performance or video conference stores the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. wired IP database; and

and a wireless mobile IP database storing the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs, wherein the client selector unit selects the wired IP database and wireless mobile IP data. At least one of the bases may be searched for, and location information and network operator information of the client IP may be obtained, and the plurality of client IPs may be selected and designated.

The additional information classification unit obtains the location information of the client corresponding to the local information by using the client's IP information, obtains the age and gender information through user subscription information, or analyzes the user's voice data to obtain the age and gender information. information can be obtained.

In order to solve the above technical problem, a method for providing an audio-tunable video service in an online performance or video conference according to an aspect of the present invention is an online performance in which a plurality of clients and a video conference server are connected through a network and the plurality of clients participate. or a method for providing a video service capable of synchronizing audio in a video conference, wherein a first client among the plurality of clients requests the video conference server for audio signals or audio signals of selected clients among the plurality of clients. requesting a support mode; receiving, by the first client, a plurality of selected client IPs corresponding to a request for a selective multi-voice support mode of the first client from the video conferencing server; requesting, by the first client, transmission of a voice signal or an audio signal from the selected plurality of clients; The first client may include receiving the voice signal or audio signal from the selected plurality of clients, mixing the video and audio signal of the online performance or video conference, and outputting the mixed signal.

In the selection of the plurality of client IPs, the videoconferencing server searches a database storing the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs to select the plurality of client IPs. there is.

In another aspect of the present invention for solving the above technical problem, a method for providing an audio-tunable video service in an online performance or video conference is an online performance in which a plurality of clients and a video conference server are connected by a network and the plurality of clients participate. In a method for providing a video service capable of synchronizing audio in a video conference or a video conference, a first client among the plurality of clients requests a video conference server for voice signals or audio signals of selected clients among the plurality of clients. requesting a mode; receiving, by the first client, a client IP list corresponding to the request for a selective multi-voice support mode of the first client from the video conferencing server; measuring, by the first client, network delay for IPs of clients in the client IP list; requesting, by the first client, transmission of a voice signal or an audio signal from a client with low network delay; and receiving, by the first client, the voice signal or audio signal from the client with low network delay, mixing the video and audio signals of the online performance or video conference, and outputting the mixed signal.

The client IP list may be generated by the video conferencing server searching a database storing the plurality of client IP information.

The first client may request the selective multi-voice support mode by further including at least one of preferred gender, age, and region information.

According to another aspect of the present invention for solving the above technical problem, a method for providing an audio-tunable video service in an online performance or video conference is provided by connecting a plurality of clients to a network around a video conference server to participate in the plurality of clients. A method for providing an audio-tunable video service in an online performance or video conference in which the video conference server acquires the plurality of client IPs, stores and classifies them together with at least one of region, age and gender information, and network delay information. ; selecting and designating, by the video conferencing server, a plurality of clients from among the plurality of clients using the classified information to reduce network delay; and the videoconferencing server receiving a voice signal or an audio signal from the designated client, mixing the video and audio signal of the online performance or video conference, and transmitting the mixed signal to the plurality of clients.

In the method for providing an audio-tunable video service in an online performance or video conference, the video conferencing server acquires location information of a client corresponding to the local information using the IP information of the client, and provides user subscription information corresponding to the client. The age and gender information may be obtained through the voice data analysis method, or the age and gender information may be obtained by analyzing user voice data.

According to an aspect of the present invention for solving the above technical problem, a video service system capable of tuning audio in an online performance or video conference includes a plurality of clients participating in an online performance or video conference and connected to a network; and a selective multi-voice support mode that is connected to the plurality of clients and provides voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients. and a video conferencing server for selecting and transmitting a plurality of client IPs corresponding to the request of the first client to the first client when a first client among the clients requests the selective multi-voice support mode. When one client receives the plurality of client IPs from the video conferencing server, it requests transmission of voice signals or audio signals from the plurality of clients corresponding to the plurality of client IPs, and transmits voice signals or audio signals from the selected plurality of clients. When receiving, it can be output by mixing with the video and audio signals of the online performance or video conference.

According to another aspect of the present invention for solving the above technical problem, an audio tunable video service system in an online performance or video conference includes a plurality of clients participating in an online performance or video conference and connected to a network; and a selective multi-voice support mode that is connected to the plurality of clients and provides voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients. and a video conferencing server generating a client IP list corresponding to the selective multi-voice support mode request of the first client and transmitting the generated client IP list to the first client when a first client among the clients requests the selective multi-voice support mode. The first client may include a client list request/receiver for receiving the client IP list from the video conferencing server by requesting the selective multi-voice support mode from the video conferencing server; a network delay measurement unit for measuring a network delay for a client corresponding to a client IP in the client IP list; a voice signal receiving unit for requesting transmission of a voice signal or audio signal from clients with low network delay and receiving the voice signal or audio signal transmitted by the client receiving the request for transmission of the voice signal or audio signal; and a client mixing unit mixing the received voice signal or audio signal with video and audio signals of the online performance or video conference and outputting the mixture.

According to another aspect of the present invention for solving the above technical problem, an audio tunable video service system in an online performance or video conference includes a plurality of clients participating in an online performance or video conference; and being connected to the plurality of clients to provide an audio-tunable video service to the plurality of clients, and to transmit voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the plurality of clients. a video conferencing server that provides a video conferencing server, wherein the video conferencing server acquires the plurality of client IPs, stores and classifies them together with at least one of region, age, gender information, and network delay information; a client selection unit which selects and designates a plurality of clients from among the plurality of clients using the classified information in order to reduce network delay; and a server mixing unit which receives voice signals or audio signals from the plurality of designated clients, mixes them with video and audio signals of the online performance or video conference, and transmits the signals to the plurality of clients.

According to the video conferencing server capable of tuning audio in an online performance or video conference according to the present invention, the video service method thereof, and the video service system using the same, an ensemble or group singing that is sensitive to the delay of audio signals in a video conference or online performance accessed by an unspecified number of people. It is possible to smoothly provide video services such as simultaneous reading of the Bible.

In other words, by mixing only the voices or audio signals of preferred participants in video conferences or online performances accessed by an unspecified number of people, the delay of real-time voice or audio signals is reduced, so real-time is important for ensemble, group singing, Bible simultaneous reading, etc. Video service can be provided seamlessly in any situation.

1 shows the configuration of a general video conference system.
2 shows the configuration of a P2P video conferencing system.
3 shows an example of the configuration of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
4 shows another example of the configuration of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
5 illustrates an implementation example of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
6 is an operational flowchart illustrating an operation of an implementation example of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
7 illustrates a configuration of a video service system capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention.
8 illustrates an implementation example of a video service system capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention.
9 is an operational flowchart illustrating an operation of an implementation example of a video service system capable of tuning audio in a broadcast-type online performance or video conference according to another embodiment of the present invention.
10 is a flowchart illustrating an example of a method for providing a video service capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
11 is a flowchart illustrating an example of a method for providing a video service capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.
12 illustrates a process in which the video conferencing server generates a client IP list corresponding to the selective multi-voice support mode and transmits it to the first client after the video conferencing server receives a request for a selective multi-voice support mode from a first client. An example for this is shown in a flow chart.
13 is a flowchart illustrating a method of providing a video service capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Since the embodiments described in this specification and the configurations shown in the drawings are only one preferred embodiment of the present invention and do not represent all of the technical spirit of the present invention, various equivalents that can replace them at the time of the present application It should be understood that there may be variations and variations.

The present invention basically reduces network delay by accepting the P2P (Peer to Peer) method. However, it is difficult to provide services to many participants with only a P2P network due to limitations in terminal and home network performance. In order to overcome the limitations of the P2P network, the present invention solves the limitations of the P2P network through selective voice provision. For example, for users singing in unison at an online performance, 『the voices of 4 to 5 female participants in their 20s with little network delay』 and the performance audio are selectively mixed and streamed in real time.

3 shows an example of the configuration of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention. Referring to FIG. 3 , a video service system capable of synchronizing audio in a P2P online performance or video conference according to an embodiment of the present invention includes a plurality of clients 320, 330, 332, 334, and 336 and a video conference server 310. ), including

The plurality of clients 320, 330, 332, 334, and 336 are users participating in online performances or video conferences, and the plurality of clients 320, 330, 332, 334, and 336 are peer to peer (P2P) through a network. peer).

The video conferencing server 310 is connected to a plurality of clients 320, 330, 332, 334, and 336 through a network, and in the online performance or video conference, the plurality of clients 320, 330, 332, 334, and 336 ), it is possible to provide a selective multi-voice support mode that provides voice signals or audio signals of selected clients to specific clients, and includes a multi-voice mode providing unit 312, a client IP selection unit 314, and a client IP transmission unit. (316).

The multi-voice mode providing unit 312 is connected to a plurality of clients participating in an online performance or video conference and connected to a network, and the audio signals or signals of clients selected from among the plurality of clients in the online performance or video conference. A selective multi-voice support mode providing an audio signal to the client is provided.

When a first client among the plurality of clients requests the selective multi-voice support mode, the client IP selector 314 selects a plurality of client IPs corresponding to the request of the first client.

The client IP transmitter 316 transmits the selected plurality of client IPs to the first client 320 .

The video conferencing server 310, for example, when the first client 320 among the plurality of clients 320, 330, 332, 334, and 336 requests the selective multi-voice support mode, the first client 320 A plurality of client IPs corresponding to the request of, for example, the second client 330, the third client 332, and the n-th client 336 are selected, and the IPs of the selected clients 330, 332, and 336 are selected. It can be transmitted to the first client 310 . Here, the first client 420 is an arbitrary client requesting the selective multi-voice support mode among multiple clients.

When the first client 320 requests the videoconferencing server 310 for the selective multi-voice support mode, the first client 320 may further include at least one of preferred gender, age, and region information in the request. At this time, the video conferencing server 310 may obtain the gender or age information through subscriber information corresponding to user information of the clients 320, 330, 332, 334, and 336 or through artificial intelligence (AI) voice analysis. there is.

The video conferencing server 310 includes at least one of a wired IP database 318 and a wireless mobile IP database 319 storing IPs of a plurality of clients 330, 332, 334, and 336 and location information of the client IPs. can be further provided. The videoconferencing server 310 may select the plurality of client IPs by obtaining location information of client IPs using the wired IP database 318 and the wireless mobile IP database 319.

In addition, the wired IP database 318 and the wireless mobile IP database 319 may store the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. At this time, the video conferencing server 310 obtains location information and network operator information of client IPs using at least one of the wired IP database 318 and the wireless mobile IP database 319 to select a plurality of client IPs. can

When the first client 320 receives the selected plurality of client IPs from the video conferencing server 310, it transmits voice signals or audio signals to the plurality of clients 330, 332, and 336 corresponding to the plurality of client IPs. When a voice signal or an audio signal is received from the selected plurality of clients 330, 332, and 336, it is mixed with the video and audio signals of the online performance or video conference and then output.

4 shows another example of the configuration of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention. Referring to FIG. 4, a video service system capable of synchronizing audio in a P2P-type online performance or video conference according to an embodiment of the present invention includes a plurality of clients 420, 430, 432, 434, and 436 and a video conference server 410. ), including

The plurality of clients 420, 430, 432, 434, and 436 are users participating in online performances or video conferences, and the plurality of clients 420, 430, 432, 434, and 436 communicate with each other through a network through peer peer).

The video conferencing server 410 is connected to a plurality of clients 420, 430, 432, 434, and 436, and selected from a plurality of clients 420, 430, 432, 434, and 436 in the online performance or video conference. A selective multi-voice support mode for providing voice signals or audio signals of clients to a specific client is provided, and when a first client 420 among the plurality of clients requests the selective multi-voice support mode, the first client 420 ) generates a client IP list corresponding to the selective multi-voice support mode request and transmits it to the first client 410. Here, the first client 420 is an arbitrary client requesting the selective multi-voice support mode among multiple clients.

The video conferencing server 410 may include a multi-voice mode provider 412, a client IP list generator 414, and a client IP list transmitter 416.

The multi-voice mode providing unit 412 is connected to a plurality of clients participating in an online performance or video conference and connected to a network, and provides voice signals or signals of clients selected from among the plurality of clients in the online performance or video conference. A selective multi-voice support mode providing an audio signal to the client is provided.

When a first client among the plurality of clients requests the selective multi-voice support mode, the client IP list generator 414 creates a client IP list corresponding to the selective multi-voice support mode request of the first client.

The client IP list transmitter 416 transmits the generated client IP list to the first client 420 .

The first client 420 may include a client list request/receiver 422, a network delay measurement unit 424, a voice signal receiver 426, and a client mixing unit 428.

The client list request/receiver 422 requests the selective multi-voice support mode from the video conference server 410 and receives the client IP list from the video conference server 410 . The network delay measuring unit 424 measures network delay for clients corresponding to the IP of the client in the client IP list. When requesting the selective multi-voice support mode, the client list request/receiver 422 may request the selective multi-voice support mode by further including at least one of preferred gender, age, and region information. At this time, the video conferencing server 410 may obtain the gender or age information through subscriber information corresponding to the user using the client or AI voice analysis.

The voice signal receiving unit 426 requests transmission of a voice signal or an audio signal from clients having a small network delay time among the measured network delays, and receives the transmitted voice signal or audio signal from the clients receiving the request for transmission of the voice signal or audio signal. receive The client mixing unit 428 mixes the received voice signal or audio signal with video and audio signals of the online performance or video conference and outputs the mixture.

The video conferencing server 410 may have a wired IP database 418 and a wireless mobile IP database 419 storing IPs of a plurality of clients 430, 432, 434, and 436 and location information of the client IPs. can The videoconferencing server 410 may obtain location information of client IPs using the wired IP database 418 and the wireless mobile IP database 419 to generate a client IP list composed of the plurality of client IPs.

In addition, the wired IP database 418 and the wireless mobile IP database 419 may store the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. At this time, the video conferencing server 410 obtains location information and network operator information of client IPs using the wired IP database 418 and the wireless mobile IP database 419, and creates a client IP list consisting of a plurality of client IPs. can create

5 shows an implementation example of a video service system capable of synchronizing audio in a P2P-type online performance or video conference according to an embodiment of the present invention, and FIG. 6 is a P2P-type online performance according to an embodiment of the present invention. It is an operation flow chart showing the operation of an implementation example of a video service system capable of tuning audio in a video conference. Referring to FIGS. 5 and 6 , an implementation example of a video service system capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention will be described.

Clients (520, 530, 540, 550) can be applications for video conferencing that run on PCs, mobile devices and the web, and video conferencing signals, media processing and audio mixing are possible. The video conferencing server 610 is a server that provides video conferencing and online performance video, and generates a client IP list by selecting clients with high transmission speeds among clients 520, 530, 540, and 550 to reduce network delay. can The online performance or religious event 510 may be a video conferencing application that runs on a PC or mobile device, and transmits video of an online performance or religious event to a client.

Client D (550) requests a selective multi-voice support mode from the video conferencing server (610). (S610) When requesting a selective multi-voice support mode, the user may add information such as preferred gender, age, and region.

The video conferencing server 610 transfers the IP list of clients meeting the user's request to Client D. (S620) Area information can be roughly obtained through IP information, and age and gender can be obtained through subscription information or through user voice data analysis.

Client D (550) measures network delay for the IP list received from the video conferencing server (610). (S630) Client D (550) requests voice transmission to clients (520, 530) with less network delay. (S640) Client A (520) and Client B (530) receiving the voice transmission request start transmitting voice signals (sounds) to Client D (550). (S650)

Client D (550) receives broadcasting (performance, religious event) video/audio from the event venue (510), and the audio signals of Client A (520), Client B (530), and Client D (550) and the event venue (510) It is mixed with broadcasting (performance, religious event) video/voice and is heard by the user through the smart phone 552. (S660)

7 illustrates a configuration of a video service system capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention. Referring to FIG. 7, a video service system capable of tuning audio in a broadcast-type online performance or video conference according to an embodiment of the present invention includes a plurality of clients 730, 732, 734, 736, and 738 and a video conference server ( 710).

The plurality of clients 730, 732, 734, 736, and 738 are users participating in online performances or video conferences, and the plurality of clients 730, 732, 734, 736, and 738 are central to the video conference server 710. are connected in a broadcast manner.

The video conferencing server 710 is connected to a plurality of clients 730, 732, 734, 736, and 738 to provide a video service capable of synchronizing audio to the plurality of clients 730, 732, 734, 736, and 738, and provides an online performance. In a video conference or a plurality of clients (730, 732, 734, 736, 738), voice signals or audio signals of selected clients are provided to the plurality of clients (730, 732, 734, 736, 738). Here, the video conferencing server 710 obtains the location information of the client corresponding to the local information using the client's IP information, obtains the age and gender information through user subscription information, or analyzes the user's voice data. The age and gender information can be obtained, and multiple clients (730, 732, 734, 736, 738) participating in online performances or video conferences can be obtained using the obtained region, age, gender information and network delay information. of clients can be selected.

The video conferencing server 710 may include an additional information classification unit 712, a client selection unit 714, and a server mixing unit 716. The additional information classification unit 712 obtains the IPs of the plurality of clients 730, 732, 734, 736, and 738, stores and classifies them together with at least one of region, age, gender information, and network delay information.

The client selector 714 selects and designates a plurality of clients with high speed among the plurality of clients 730 , 732 , 734 , 736 , and 738 using the classified information in order to reduce network delay. The server mixing unit 716 receives voice signals or audio signals from the plurality of designated clients, mixes them with video and audio signals of the online performance or video conference, and provides information to the plurality of clients 730, 732, 734, 736, and 738. send.

The video conferencing server 710 stores the IPs of the plurality of clients 730, 732, 734, 736, and 738 and the location information of the client IPs in a wired IP database 720 and a wireless mobile IP database 725. can be provided The videoconferencing server 710 may acquire location information of client IPs using the wired IP database 720 and the wireless mobile IP database 725 to select the plurality of client IPs.

In addition, the wired IP database 720 and the wireless mobile IP database 725 may store the plurality of client IPs, location information of the client IPs, and network operator information of the client IPs. At this time, the video conferencing server 710 may select a plurality of client IPs by obtaining location information and network operator information of client IPs using the wired IP database 720 and the wireless mobile IP database 725. At this time, the client selector 714 searches the wired IP database 720 or the wireless mobile IP database 725 to obtain location information and network operator information of the client IP, selects and assigns the plurality of client IPs. can

8 shows an implementation example of a video service system capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention, and FIG. It is an operational flowchart showing an operation of an implementation example of a video service system capable of tuning audio in a cast-type online performance or video conference. Referring to FIGS. 8 and 9 , an implementation example of a video service system capable of tuning audio in a broadcast-type online performance or video conference according to another embodiment of the present invention will be described.

Clients (820, 830, 840, 850) can be applications for video conferencing that run on PCs, mobile devices and the web, and video conferencing signals, media processing and audio mixing are possible. The video conferencing server 910 is a server that provides video conferencing and online performance video, and can designate clients with high transmission speeds in order to reduce network delay. The performance and religious event venue 810 can be a video conferencing application that runs on PCs and mobile devices, and transmits videos of online performances and religious events to the video conferencing server 910.

The video conferencing server 910 relays a performance video or religious event video and delivers it to the clients 820, 830, 840, and 850 in real time. (S910)

And the video conferencing server 910 obtains user IP information, stores and classifies information such as region, age, gender information, and network delay information. Area information can obtain approximate location information through IP information, and age and gender information can be obtained through subscription information or through user voice data analysis.

At this time, the video conferencing server 910 selects a plurality of fast clients (eg, 820, 830) from among the plurality of clients 820, 830, 840, and 850 using the classified information to reduce network delay. After designation, voice signals or audio signals may be received from the designated clients 820 and 830, mixed with video and audio signals of the online performance or video conference, and transmitted to the plurality of clients.

Meanwhile, a method for providing a video service capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention will be described.

10 is a flowchart illustrating an example of a method for providing a video service capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention.

Referring to FIGS. 3 and 10, a method of providing a video service capable of synchronizing audio in an online performance or video conference in which a plurality of clients and a video conference server are connected through a network will be described. First, among a plurality of clients (320, 330, 332, 334, 336), a first client (320) sends a video conferencing server (310) a selected number of clients (320, 330, 332, 334, 336). A selective multi-voice support mode requesting voice signals or audio signals of clients is requested (step S1010). When the first client 310 requests a selective multi-voice support mode, among preferred gender, age, and region information The selective multi-voice support mode may be requested by including at least one more.

When the video conferencing server 310 receives a request for a selective multi-voice support mode from the first client 320 and selects and transmits a plurality of client IPs corresponding to the selective multi-voice support mode, the first client 320 ) receives the selected plurality of client IPs. (Step S1020) Here, for the selection of the plurality of client IPs, the video conferencing server 310 determines the IPs of the plurality of clients 320, 330, 332, 334, and 336. , Searching the wired IP database 318 and the wireless mobile IP database 319 storing the location information of the client IP and the network operator information of the client IP, obtaining the location information of the client IP and the network operator information IPs of multiple clients with high speed can be selected.

The first client 320 receives IP information of a plurality of clients selected from the video conferencing server 310, and requests transmission of audio signals or audio signals from clients corresponding to the IP information of the plurality of clients. ( Step S1030)

When the selected plurality of clients transmit a voice signal or an audio signal corresponding to a voice signal or an audio signal transmission request from the first client to the first client 320, the first client 320 responds to the selected plurality of clients. Receives a voice signal or an audio signal from (Step S1040)

When the first client 320 receives the voice signal or audio signal from the selected plurality of clients, the first client 320 mixes and outputs the video and audio signal of the online performance or video conference, so that the user can hear the synchronized video of the voice signal. (Step S1050)

11 is a flowchart illustrating an example of a method for providing a video service capable of tuning audio in a P2P-type online performance or video conference according to an embodiment of the present invention. Referring to FIGS. 4 and 11, a plurality of clients 420, 430, 432, 434, and 436 and a video conferencing server 410 are connected to a network, and a plurality of clients 420, 430, 432, 434, and 436 participate. A method for providing an audio-tunable video service in an online performance or video conference is described.

First, the first client 420 among the plurality of clients 420, 430, 432, 434, and 436 performs a video conference in a selective multi-voice support mode in which a first client 420 requests voice signals or audio signals of clients selected from among the plurality of clients. (Step S1110) At this time, the first client 420 may request the selective multi-voice support mode by further including at least one of preferred gender, age, and region information.

When the video conferencing server 410 generates a client IP list corresponding to the selective multi-voice support mode request of the first client 420 and transmits it to the first client 410, the first client 420 responds to the client IP list. The list is received. (Step S1120) Here, the creation of the client IP list may be performed by the video conferencing server 410. Specifically, the video conferencing server 410 stores the IPs of the plurality of clients 430, 432, 434, and 436 and the location information of the client IPs in a wired IP database 418 and a wireless mobile IP database 419. can be provided At this time, the videoconferencing server 410 may obtain location information of client IPs using the wired IP database 418 and the wireless mobile IP database 419 and generate a client IP list consisting of the plurality of client IPs. there is.

In addition, when the wired IP database 418 and the wireless mobile IP database 419 store a plurality of client IPs, location information of the client IPs, and network operator information of the client IPs, the videoconferencing server 410 The client IP list may be created using location information of client IPs obtained by searching the IP database 418 and the wireless mobile IP database 419 and network operator information.

When the first client 410 receives the client IP list from the video conferencing server 410, it measures network delay for the client IPs in the client IP list. (Step S1130)

As a result of the measurement, the first client 410 requests transmission of a voice signal or an audio signal from clients with low network delay (step S1140).

When the clients receiving the voice signal or audio signal transmission request transmit the voice signal or audio signal to the first client 420, the first client 420 receives the voice signal or audio signal. (Step S1150)

When the first client 410 receives a voice signal or an audio signal from the clients, the first client 410 can hear the tuned voice to the user by mixing and outputting the video and audio signal of an online performance or video conference. (Step S1060)

12 shows that after the video conferencing server 410 receives a request for a selective multi-voice support mode from the first client 420, the video conferencing server 410 performs a client IP corresponding to the selective multi-voice support mode in step S1120. An example of a process of generating a list and transmitting it to the first client 420 is shown in a flowchart.

When the videoconferencing server 410 receives a request for a selective multi-voice support mode from client 1 420, it can create and transmit a client IP list corresponding to the request. The client IP list can be created by checking the IPs corresponding to conditions 1, 2, and 3 above, with condition 1 being gender/age/region, condition 2 being the same network operator, and condition 3 being a nearby IP. At this time, the location information may be obtained by utilizing the user's IP information. Specifically, a location can be estimated using a wired IP DB and a wireless mobile IP DB.

And gender/age information can be obtained by using subscriber information or AI voice analysis technology. In addition, since communication between subscribers of the same network operator is faster, location information and whether or not they are the same network operator can be used to create a client IP list.

Creating a client IP list will be described as an example. Referring to FIG. 12, the video conferencing server 410 receives a request for a selective multi-voice support mode (step S1210). At this time, the first client 420 sets a preferred voice condition (gender, age, age). can be sent together.

The video conferencing server 410 may include a wired IP database 418 and a wireless mobile IP database 419, and the wired IP database 418 and the wireless mobile IP database 419 may include a plurality of client IPs, the When the location information of the client IP and network operator information of the client IP are stored, the videoconferencing server 410 searches the wired IP database 418 and the wireless mobile IP database 419 to obtain the location of the client IP. The client IP list may be generated using information and network operator information.

Specifically, the video conferencing server 410 searches the wired IP database 418 and the wireless mobile IP database 419 to find the preferred voice condition (gender/age/region, condition 1) transmitted by the first client 420. ) to obtain the corresponding IPs. (Step S1220) and the video conferencing server 410 obtains IPs corresponding to the same network operator IP (condition 2) through a database (418, 419) search. (Step S1230) In addition, the videoconferencing server obtains IPs corresponding to nearby local IPs (condition 3) through a search of the databases 418 and 419. (Step S1240)

The videoconferencing server 410 creates a client IP list satisfying all conditions 1, 2, and 3 above (step S1250) and delivers the client IP list to the first client 420. (Step S1280)

If there is no client IP that satisfies all of conditions 1, 2, and 3, the video conferencing server 410 creates a list of client IPs that satisfy conditions 1 and 2 (step S1260), and satisfies conditions 1 and 2 above. and transfers the client IP list to the first client 420. (Step S1280)

If there is no client IP that satisfies conditions 1 and 2, the videoconferencing server 410 creates a client IP list that satisfies condition 1 (step S1270), and converts the client IP list that meets condition 1 into a first It is delivered to the client 420. (Step S1280)

If there is no client IP that satisfies condition 1, the video conferencing server 410 may create a client IP list of client IPs searched for under a preset condition and deliver the result to the first client 420. (Step S1280)

13 is a flowchart illustrating a method of providing a video service capable of tuning audio in a broadcast-based online performance or video conference according to another embodiment of the present invention. Referring to FIGS. 7 and 13 , a method for providing a video service capable of tuning audio in a broadcast-type online performance or video conference according to another embodiment of the present invention will be described.

A plurality of clients (730, 732, 734, 736, 738) are connected to the video conferencing server (710) through a network to perform an online performance or video conference in which the plurality of clients (730, 732, 734, 736, 738) participate. In order to provide an audio-tunable video service, the video conferencing server 710 acquires the IPs of a plurality of clients 730, 732, 734, 736, and 738, and at least one of region, age, gender information, and network delay information. Save and classify with (Step S1310) At this time, the video conferencing server 710 obtains the location information of the client corresponding to the local information using the IP information of the client, and the age and gender information through the user subscription information corresponding to the client. The age and gender information may be obtained by acquiring or analyzing the user's voice data.

The video conferencing server 710 selects and designates a plurality of fast clients among the plurality of clients 730, 732, 734, 736, and 738 using the classified information to reduce network delay. (Step S1320)

The video conference server 710 requests a voice signal or audio signal from the designated clients (step S1330). When the video conference server 710 receives a voice signal or audio signal from the designated clients, the online performance or video conference video and audio signals are mixed and transmitted to the plurality of clients. (Step S1340)

On the other hand, Table 1 summarizes the advantages and disadvantages of the general video conference, P2P, clubhouse method and the method of the present invention. Referring to Table 1, the network delay is the highest in general video conferencing, and the other P2P methods, clubhouse methods, and the methods of the present invention are low. The number of meeting participants is high in general video conferencing, small in P2P, and medium in the clubhouse method and the method of the present invention. Regarding audio mixing, the general video conference and P2P method mixes audio for all meeting participants, the clubhouse method mixes audio only for those who have the right to speak, and the method of the present invention mixes only voice signals that I prefer and have low network delay do

[Table 1]

And, in the present invention, the client can stream in real time by mixing “videos and audios of performances and religious events” and “audio signals directly requested and received by the client”. Table 2 shows a comparison of broadcast streaming, video conferencing, online ensemble, club house, and the case of the present invention.

[Table 2]

Although the present invention has been described with reference to the embodiments shown in the drawings, this is only exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached claims.

316: client IP transmission unit
318, 418, 720: wired IP database
319, 419, 725: wireless mobile IP database
320, 420, 730: first client
330, 430, 732: second client
332, 432, 734: 3rd client
334, 434, 736: n-1 client
336, 436, 738: nth client
412: multiple voice mode providing unit
414: client IP list generation unit
416: client IP list transmission unit
422: client list request/receiver
424: network delay measuring unit
436: voice signal receiver
438: client mixing unit
510, 810: Venue
520, 820: Client A
530, 830: Client B
540, 840: Client C
550, 850: Client D
552: smartphone
610, 910: video conferencing server
712: additional information classification unit
714: client selection unit
716: server mixing unit

Claims (20)

Selection that provides voice signals or audio signals of clients selected from among the plurality of clients participating in an online performance or video conference, connected to a network, and selected from among the clients in the online performance or video conference to the clients a multi-voice mode provider providing multiple voice support modes;
a client IP selector for selecting a plurality of client IPs corresponding to the request of the first client when a first client among the plurality of clients requests the selective multi-voice support mode; and
a client IP transmission unit for transmitting the selected plurality of client IPs to the first client;
Here, when the first client receives the plurality of client IPs, the first client requests transmission of voice signals or audio signals from the plurality of clients corresponding to the plurality of client IPs, and receives voice signals or audio signals from the selected plurality of clients. A video conferencing server that provides an audio-tunable video in an online performance or video conference, mixing and outputting the video and audio signals of the online performance or video conference. According to claim 1,
a plurality of client IPs corresponding to the plurality of clients and at least one of a wired IP database and a wireless mobile IP database storing location information of the plurality of client IPs;
The client IP selector
Provides a video service capable of tuning audio in an online performance or video conference, characterized in that the plurality of client IPs are selected by obtaining location information of the client IPs using at least one of the wired IP database and the wireless mobile IP database. Video conferencing server. According to claim 1,
At least one of a wired IP database and a wireless mobile IP database storing a plurality of client IPs corresponding to the plurality of clients, location information of the plurality of client IPs, and network operator information of the client IPs;
The client IP selector
Audio tunable video in an online performance or video conference, characterized in that the plurality of client IPs are selected by obtaining location information and network operator information of the client IP using at least one of the wired IP database and the wireless mobile IP database A video conferencing server that provides The method of claim 1, wherein the client IP selection unit
Upon receipt of a request for the selective multi-voice support mode including at least one of preferred gender, age, and region information from the first client, obtaining the gender or age information through subscriber information or artificial intelligence (AI) voice analysis Characterized in that, a video conference server that provides audio tunable video in online performances or video conferences. In a video conference server that provides audio tunable video in an online performance or video conference,
Selection that provides voice signals or audio signals of clients selected from among the plurality of clients participating in an online performance or video conference, connected to a network, and selected from among the clients in the online performance or video conference to the clients a multi-voice mode provider providing multiple voice support modes;
a client IP list generation unit generating a client IP list corresponding to the selective multi-voice support mode request of the first client when a first client among the plurality of clients requests the selective multi-voice support mode; and
an IP list transmission unit for transmitting the generated client IP list to the first client;
Here, the first client requests the videoconferencing server for the selective multi-voice support mode, receives the client IP list from the videoconferencing server, and targets the client corresponding to the client IP in the client IP list through the network. When a voice signal or audio signal is received by measuring a delay and requesting transmission of a voice signal or audio signal from clients with low network delay, mixing with the video and audio signal of the online performance or video conference and outputting the online A video conferencing server that provides audio-synchronizable video during performances or video conferences. According to claim 5,
Further comprising at least one of a wired IP database and a wireless mobile IP database storing a plurality of client IPs corresponding to the plurality of clients and location information of the client IPs;
The client IP list generating unit
A video conference that provides audio tunable video in an online performance or video conference, characterized in that the client IP list is generated by obtaining the location information of the client IP using at least one of the wired IP database and the wireless mobile IP database. 's server. According to claim 5,
At least one of a wired IP database and a wireless mobile IP database storing a plurality of client IPs corresponding to the plurality of clients, location information of the client IPs, and network operator information of the client IPs;
The client IP list generating unit
Characterized in that the client IP list is generated by obtaining location information and network operator information of the client IP using at least one of the wired IP database and the wireless mobile IP database, A video conferencing server that provides The method of claim 5, wherein the client IP list generating unit
Upon receipt of a request for the selective multi-voice support mode including at least one of preferred gender, age, and region information from the first client, obtaining the gender or age information through subscriber information or artificial intelligence (AI) voice analysis Characterized in that, a video conference server that provides audio tunable video in online performances or video conferences. A video conference server connected to a plurality of clients participating in an online performance or video conference and providing an audio-tunable video service to the plurality of clients,
an additional information classifying unit that obtains a plurality of client IPs corresponding to the plurality of clients, stores them together with at least one of region, age, gender information, and network delay information, and classifies them;
a client selection unit that selects and designates a plurality of clients with high speeds from among the plurality of clients using the classified information to reduce network delay; and
A server mixing unit that receives voice signals or audio signals from the plurality of designated clients, mixes them with video and audio signals of the online performance or video conference, and transmits them to the plurality of clients; A video conferencing server that provides video. 10. The method of claim 9, wherein the additional information classification unit
Obtaining the client's location information corresponding to the area information using the client's IP information, obtaining the age and gender information through user subscription information, or acquiring the age and gender information by analyzing the user's voice data. Characteristically, a video conferencing server that provides audio-tunable video in online performances or video conferences. A method for providing an audio-tunable video service in an online performance or video conference in which a plurality of clients and a video conference server are connected by a network and the plurality of clients participate,
requesting, by a first client among the plurality of clients, a selective multi-voice support mode requesting voice signals or audio signals of selected clients from among the plurality of clients to the video conferencing server;
receiving, by the first client, a plurality of selected client IPs corresponding to a request for a selective multi-voice support mode of the first client from the video conferencing server;
requesting, by the first client, transmission of a voice signal or an audio signal from the selected plurality of clients;
The first client receives the voice signal or audio signal from the selected plurality of clients, mixes it with the video and audio signals of the online performance or video conference, and outputs the mixed audio synchronization in an online performance or video conference. Possible video service delivery methods. 12. The method of claim 11, wherein the selection of the plurality of client IPs
The video conferencing server selects the plurality of client IPs by searching a database storing a plurality of client IPs corresponding to the plurality of clients, location information of the plurality of client IPs, and network operator information of the client IPs. Characterized in that, a method of providing an audio tunable video service in an online performance or video conference. A method for providing an audio-tunable video service in an online performance or video conference in which a plurality of clients and a video conference server are connected by a network and the plurality of clients participate,
requesting, by a first client among the plurality of clients, a selective multi-voice support mode requesting voice signals or audio signals of selected clients from among the plurality of clients to the video conferencing server;
receiving, by the first client, a client IP list corresponding to the request for a selective multi-voice support mode of the first client from the video conferencing server;
measuring, by the first client, network delay for IPs of clients in the client IP list;
requesting, by the first client, transmission of a voice signal or an audio signal from a client with low network delay; and
The first client receives the voice signal or audio signal from the client with low network delay, mixes it with the video and audio signals of the online performance or video conference, and outputs the mixed audio signal. How to provide video service. 14. The method of claim 13, wherein the client IP list is
characterized in that the video conferencing server is generated by searching for a database storing a plurality of client IP information corresponding to the plurality of clients, and providing a video service capable of tuning audio in an online performance or video conference. According to claim 11 or 13,
Wherein the first client requests the selective multi-voice support mode by further including at least one of preferred gender, age, and region information. A method for providing an audio-tunable video service in an online performance or video conference in which a plurality of clients are connected to a network around a video conference server, the method comprising:
obtaining, by the videoconferencing server, the plurality of client IPs, storing and classifying them together with at least one of region, age, gender information, and network delay information;
selecting and designating, by the video conferencing server, a plurality of clients from among the plurality of clients using the classified information to reduce network delay; and
The video conferencing server receives a voice signal or audio signal from the designated client, mixes it with the video and audio signals of the online performance or video conference, and transmits the audio to the plurality of clients. How to provide tunable video service. According to claim 16,
The videoconferencing server obtains the location information of the client corresponding to the local information using the IP information of the client, obtains the age and gender information through user subscription information corresponding to the client, or analyzes the user's voice data. A method for providing an audio-tunable video service in an online performance or video conference, characterized in that by obtaining the age and gender information by doing so. A plurality of clients participating in online performances or video conferences and connected to a network; and
Provides a selective multi-voice support mode that is connected to the plurality of clients and provides voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients; a video conferencing server for selecting and transmitting a plurality of client IPs corresponding to the request of the first client to the first client when the first client requests the selective multi-voice support mode;
When the first client receives the plurality of client IPs from the video conferencing server, the first client requests transmission of voice signals or audio signals from the plurality of clients corresponding to the plurality of client IPs, and transmits voice signals or audio signals from the selected plurality of clients. A video service system capable of synchronizing audio in an online performance or video conference, characterized in that when an audio signal is received, it is mixed with the video and audio signals of the online performance or video conference and then output. A plurality of clients participating in online performances or video conferences and connected to a network; and
Provides a selective multi-voice support mode that is connected to the plurality of clients and provides voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the clients; a video conferencing server generating a client IP list corresponding to the selective multi-voice support mode request of the first client and transmitting it to the first client when the first client requests the selective multi-voice support mode;
1st client
a client list request/receiver requesting the selective multi-voice support mode from the video conference server and receiving the client IP list from the video conference server;
a network delay measurement unit for measuring a network delay for a client corresponding to a client IP in the client IP list;
a voice signal receiving unit for requesting transmission of a voice signal or audio signal from clients with low network delay and receiving the voice signal or audio signal transmitted by the client receiving the request for transmission of the voice signal or audio signal; and
and a client mixing unit for mixing the received voice signal or audio signal with the video and audio signals of the online performance or video conference and outputting the mixed audio signal. Multiple clients participating in online performances or video conferencing; and
It is connected to the plurality of clients to provide an audio-tunable video service to the plurality of clients, and provides voice signals or audio signals of clients selected from among the plurality of clients in the online performance or video conference to the plurality of clients. Including a video conferencing server that
The video conferencing server
an additional information classifying unit that obtains the plurality of client IPs, stores them together with at least one of region, age, gender information, and network delay information, and classifies them;
a client selection unit which selects and designates a plurality of clients from among the plurality of clients using the classified information in order to reduce network delay; and
A server mixing unit that receives voice signals or audio signals from the plurality of designated clients, mixes them with video and audio signals of the online performance or video conference, and transmits them to the plurality of clients; Video service system.

Images