KR102467081B1 - Video review system, method and program - Google Patents

Abstract

In the video review method according to an embodiment of the present invention, when a user inputs a video file to be checked and reviewed, the video review program executed on the screen of the user terminal analyzes the video file and extracts it into video data and audio data. step; The video review program extracts at least one video still picture for each section from the extracted video data in units of a review time interval input by a user and outputs the still picture to the screen of the user terminal; generating, by the video review program, subtitle information and volume information for each section corresponding to the video still screen for each section separately from the extracted audio data; and outputting the video still screen for each section, subtitle information, and volume information by classifying the video still screen of the corresponding part to the screen of the user terminal.

Description

Video review system, method and program {Video review system, method and program}

The present invention relates to a video review system and method, and more particularly, to a video review system and method for efficiently checking whether a video has a problem by providing a still image, audio content, and volume information at a time interval input by a user. and about the program.

In a situation where various video contents such as education, culture, and hobbies increase exponentially, the task of ensuring that the videos that need to be uploaded, maintained, and managed are properly organized in the system and video content platforms such as YouTube and VIMEO becomes vast and difficult. lost.

In order to check whether there is a problem with the video, the user or the system administrator must play the video one by one and check it.

Since this task is very time-consuming and cumbersome in a system that needs to manage many videos, a technique for quickly reviewing the entire video content is required to efficiently check a large amount of videos with a long playing time.

One of the ways to quickly check whether there is a problem with the video and whether the video is properly composed is to display the screen of the video at regular intervals and output audio information of the corresponding part.

Therefore, in order to solve the above problems, research on a video review system and method has been required.

Korean Registered Patent No. 10-1682076 (registered on November 28, 2016)

An object of the present invention is to provide a video review system and method that efficiently checks whether a video has a problem by providing a still image, audio content, and volume information at a time interval input by a user.

In the video review method according to an embodiment of the present invention, when a user inputs a video file to be checked and reviewed, the video review program executed on the screen of the user terminal analyzes the video file and extracts it into video data and audio data. step; The video review program extracts at least one video still picture for each section from the extracted video data in units of a review time interval input by a user and outputs the still picture to the screen of the user terminal; generating, by the video review program, subtitle information and volume information for each section corresponding to the video still screen for each section separately from the extracted audio data; and outputting the video still screen for each section, subtitle information, and volume information by classifying the video still screen of the corresponding part to the screen of the user terminal.

The video review program inputs a video playback screen, a menu related to video playback, and a video information unit displaying information on a video file to display information about the video, and when reviewing a video displayed in the video information unit, a corresponding video review time interval. A time display unit for displaying a subtitle output time to check where the currently output subtitle corresponds to a playback time interval of a video, and a time interval for reviewing a video input from the time display unit when reviewing a video displayed in the video information unit. and a section information unit for displaying section information including at least one video still screen, subtitle information, and volume information for each corresponding section.

the video review program extracting audio data through analysis of a video file to be reviewed; The video review program generates PCM data for each section according to the review time interval and subtitle output time input by the user, and transmits the audio data to a service providing server; the service providing server inputting the received PCM data to the STT engine, generating an STT conversion result, and transmitting the result to a user terminal where the video review program is executed; The video review program further includes displaying and providing subtitle information for each segment corresponding to the video review time interval by dividing and outputting the STT conversion result by segment.

It is performed by a video review program stored in a computer-readable storage medium that performs the above video review method.

In the video review system using the video review method according to an embodiment of the present invention, it is connected to a user terminal where the video review program is executed through a communication network, and the video review program executed in the user terminal is connected to the audio extracted from the video to be reviewed. and a service providing server having an STT engine for performing subtitle text conversion for the corresponding voice when requesting subtitle text conversion.

The service providing server includes a communication unit having at least one communication protocol for communicating with a user terminal through a communication network and performing data communication of information necessary to provide a video review service; a management unit having an STT engine to perform audio-caption text conversion on the video to be reviewed, and transmitting the STT conversion result to the user terminal through the communication unit; Performs a control function for the user interface implemented in the video review program, plays a video from a still video screen or plays volume information according to a user input operation through the video review program, and performs a user input operation for subtitle information. , a display control unit providing an interface so that the text of the corresponding subtitle information can be directly re-entered and corrected.

As the service providing server provides a video review program, it collects video review details and manages the information so that it can be viewed upon user request. In addition to including, it further includes user evaluation or satisfaction with the video review results, and machine learning-based learning can be performed using the collected video review history, and as a result of learning, it is possible to detect problems or errors in the video review program. Further includes a statistical learning unit that determines the result for and provides it to the manager.

The present invention classifies and provides video still images, subtitle information, and audio information for each section so that video review can be efficiently performed, and provides a function to detect and immediately correct errors in video subtitles and audio in the section. By doing so, not only user convenience is increased, but also various video contents can be reviewed, which is advantageous in creating demand for business related to video contents, which are rapidly growing.

In addition, the present invention has the advantage of being efficient in reviewing video errors by applying a multi-angle review method for each section to a section-by-section still screen, audio (including volume status), subtitles, etc. through a split screen, even for a vast amount of video. .

1 is a block diagram showing the configuration of a video review system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the internal configuration of the video review system of FIG. 1 in detail.
3 is a flowchart of a video review method according to an embodiment of the present invention.
4 is a flowchart illustrating a process of converting audio data in detail in a video review method according to an embodiment of the present invention.
5 is a diagram showing a screen of a video review program implementing a video review method according to an embodiment of the present invention by way of example.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. However, the spirit of the present invention is not limited to the presented embodiments, and those skilled in the art who understand the spirit of the present invention may add, change, delete, etc. other elements within the scope of the same spirit, through other degenerative inventions or the present invention. Other embodiments included within the scope of the inventive idea can be easily proposed, but it will be said that this is also included in the scope of the inventive concept. In addition, components having the same function within the scope of the same idea appearing in the drawings of each embodiment are described using the same reference numerals.

Figure 3 is a flow chart of a video review method according to an embodiment of the present invention, Figure 4 is a flow chart showing the audio data conversion process in detail in the video review method according to an embodiment of the present invention, Figure 5 is a flow chart of the present invention It is a diagram showing an example of a video review program screen implementing a video review method according to an embodiment.

A video review method according to an embodiment of the present invention controls audio and video separation of a video to be provided to the user terminal 200 for video review through a video review program executed on the screen of the user terminal 200 and conversion of audio subtitles. and processing information about the displayed section.

At this time, referring to FIG. 5, the video review program can implement a user interface on the program screen so that it is divided into three areas and displayed, and the three areas are a video information unit (V), a time display unit (T) and a section information unit ( I) consists of

The video information unit (V) provides information about the video to be displayed, such as video screen, video playback related menu (play, stop, pause, etc.), video file information (title, file size, file format, playback time, etc.) ) and can be displayed on the left screen of the section information unit (I) so that intuitive confirmation of each video section is possible.

The time display unit (T) provides a way to check the section to be reviewed and the output time of subtitles when reviewing a video, and the location is provided in the form of a bar at the top of the video information section (V) and section information section (I) so that the user can output subtitles during review. You can check the time and provide an easy way to enter the video review time interval you want to review.

The section information unit (I) provides section information including a video still screen, subtitle information, and volume information for each section for the video review interval input when reviewing the video.

Section information for at least one section may be determined by dividing the section of the video review interval into a predetermined time interval or image frame interval within the review interval.

Also, the video still image may be a still image for a video frame of a corresponding section.

In addition, the subtitle information and volume information may include the subtitle and volume data of the corresponding section, the subtitle information is displayed and provided in text form, and the volume information is provided in the form of a button so that a user input operation for the volume information button (eg, mouse An input operation such as a click or a touch, etc.) may be provided so that the audio of the corresponding section is output.

In addition, the video still screen can be controlled to be played from the still screen for the video frame of the corresponding section when the user inputs, and through this, the video, subtitle, and volume of each section are reviewed together, so that the user can easily view the subtitles and subtitles for each section of the video. Since the volume can be matched and viewed, it is easy to find the wrong part during review.

Specifically, the user of the video review program can input a time interval to review the video, and the video review program can check the still image of the corresponding part, subtitle content for a certain period of time, and volume information according to the input time interval.

The user checks subtitles and volume information for each section, and if it is determined that there is a problem, the user can press the still screen and play the video of the corresponding part to check in detail.

The information provided to the user or manager by the video review program is largely the start and stop screen of the corresponding section of the video, subtitle information including subtitle content, and volume information.

5 is a process showing how video-related information to be provided is processed.

After analyzing the video file to be checked by dividing it into video and audio parts, still images are extracted from the video-related data in units of time input by the user and displayed on the screen. In audio-related data, section caption information and volume information are generated and output together with a still image of the corresponding section.

Now, with reference to the process of FIG. 3, a video/audio analysis and conversion process for reviewing each video section will be described.

First, when a user inputs a video file to be checked and reviewed, the video review program analyzes the video file and divides it into a video part and an audio part and extracts them (S100 to S104).

From the extracted video data, a video still screen for each section is extracted in units of time input by the user and output to the screen of the user terminal 200 (S106).

In addition, in the audio data, subtitle information and volume information for each section corresponding to the video review time interval input by the user are separately generated, and as shown in FIG.

A detailed processing process for generating subtitles of audio data is shown in FIG. 4, and audio contents can be converted into subtitles and provided through Speech-to-Text (STT) technology, which is a technology for converting audio data into text.

In order to convert audio into subtitles, a service providing server 100 on a communication network 300 equipped with an STT engine is provided. In addition, the user terminal 200 may execute a video review program, transmit audio data extracted from the video to the service providing server 100, and receive an STT conversion result, that is, subtitle information.

Specifically, referring to FIG. 4 , the video review program first extracts audio data through analysis of a video file to be reviewed (S200 and S202).

The video review program generates Pulse-Code Modulation (PCM) data for each section according to the review time interval and subtitle output time input by the user and transmits the audio data to the service providing server 100 (S204 and S206).

The service providing server 100 inputs the received PCM data to the STT engine, generates an STT conversion result, and transmits it to the user terminal 200 where the video review program is executed (S208).

The video review program divides and outputs the STT conversion result by section, and displays and provides subtitle information to the user by section corresponding to the video review time interval (S210).

In addition, at this time, when the user terminal 200 requests the STT engine for STT conversion of the audio extracted from the video to be reviewed, the desired country language can be selected and the language information to be converted can be transmitted together, and the STT engine converts to the corresponding language STT conversion results are provided, and multiple conversion results may be provided upon request for multiple languages.

Furthermore, the subtitle information for each section checked by the user terminal 200 may have an error according to the conversion result, and by checking the corresponding audio information for each section and directly modifying and converting it, the user can easily access a specific section. Subtitle corrections can be made.

In addition, caption information and video still images for each section may be matched to check whether they correspond to the caption of the corresponding section, so that caption sync is adjusted.

1 is a block diagram showing the configuration of a video review system according to an embodiment of the present invention, and FIG. 2 is a block diagram showing the internal configuration of the video review system of FIG. 1 in detail.

The video review system of the present invention includes a user terminal 200 in which a video review program is executed, and a service providing server 100 connected through a communication network 300 and having an STT engine for converting subtitles to audio into text. include

The user terminal 200 is a personal terminal capable of executing a video review program, and may be, for example, a portable terminal including a PC, a laptop computer, a tablet computer, a smart phone, and a phablet phone.

In addition, the service providing server 100 has an STT engine, and when a video review program executed in the user terminal 200 requests subtitle text conversion of audio extracted from a video to be reviewed, STT conversion is performed and the conversion result is transmitted to the user terminal ( 200) side.

Furthermore, the service providing server 100 may provide a service to provide a video review program based on the web in addition to the method installed in the user terminal 200 .

In addition, the service providing server 100 has a detailed configuration for providing a program service in a user terminal 200 installation method or a web-based method, and as shown in FIG. 2, the communication unit 110, the management unit 120, and the display control unit 130 , may further include a statistical learning unit 140 and a database 150.

The communication unit 110 communicates with the user terminal 200 through the communication network 300, performs data communication of information necessary to provide a video review service, and includes one or more compatible communication protocols of the communication network 300 for this purpose. can

The management unit 120 is provided with an STT engine to perform audio-caption text conversion on the video to be reviewed, and transmits the conversion result (caption information including caption text) to the user terminal 200 through the communication unit 110. do.

In addition, the management unit 120 may further include a function for providing a program service when providing a web-based video review program. For example, it may receive member information and perform program management for each member.

In addition to providing program services through a membership system, a trial version and a full version by paid payment are provided, and when using the full version service, payment is made for each log-in or a fixed-term payment method may be employed.

In addition, when the program is operated on a web basis in the service providing server 100, server login may be attempted.

The first member sign-up process is required for login authentication, and when proceeding as a non-member, the program may be used temporarily by personal information authentication, but benefits provided to members such as discounts may be reduced.

Login authentication may be performed by matching with personal information provided at the time of initial member registration, and for this purpose, the service providing server 100 may encrypt and store the personal information in the database 150.

When login authentication is completed, the program is provided to be executed in the user terminal 200 in a web-based form through the communication network 300 .

Furthermore, integrity verification of authentication is performed during login authentication, and member information management and storage based on blockchain can be performed to protect individual member information.

Specifically, a blockchain network is established in conjunction with a number of blockchain servers, public keys and private keys are generated through the established internal blockchain network, converted into hash values, distributed and stored, and distributed public keys and users User login authentication can be performed based on the personal information of the user.

Furthermore, each user certificate including a hash value for the user information may be generated by receiving individual user information along with a public key from the plurality of user terminals 200, and the storage method for each user certificate is a merkle tree. structure can be achieved.

For example, each user certificate (transaction) is stored with a hash value in the lowest child node, and the hash value is stored in the intermediate node on the path leading to the lowest child node in the merkle root (parent node), which is the top level of the merkle tree. It is hashed and stored so that it can be shared.

When determining the authenticity of the stored user certificate through this, the user certificate copied to the individual user terminal 200 and the user certificate in the database 150 are compared, and only hash values that have been hashed along the path of the Merkle tree are compared. It is done by

At this time, since the comparison operation is performed on the path of the Merkle tree, it is not necessary to perform the comparison operation on the blocks of all nodes, so it is possible to determine the authenticity with a relatively easy amount of calculation, and forgery of the transaction can be found easily and quickly. , Transactions can be easily verified even in the user terminal 200 in the form of a portable terminal with a small capacity.

The display control unit 130 performs a control function for the user interface implemented in the video review program, and specifically controls to configure the program screen as shown in FIG. 5, displays the video information unit (V), time display unit (T), and section As described above in the information unit (I), according to the user's input operation, the video can be played from the still screen of the video or volume information can be reproduced. In addition, when a user input operation is performed on the caption information, an interface may be provided so that the text of the corresponding caption information can be directly re-entered and corrected.

As the statistical learning unit 140 provides a video review program for each member, it collects the video review details for each member, stores them in the database 150, and manages the information so that the member can view the information upon request. The review history may further include a user's evaluation or satisfaction with respect to the video review result, as well as the contents of the video correction, the corresponding video information (information including subtitles, volume information, etc.), and the like.

Furthermore, the statistical learning unit 140 may perform machine learning-based learning using the collected video review details, and may determine problems or errors of the video review program as learning results and provide the information to a manager.

For example, when learning machine learning, a standard value for error or problem determination is set for a specific review area (subtitle, audio, video playback section, etc.), and the number of times problems are raised when a problem is raised in the user evaluation or satisfaction level after review According to this, it is determined whether the reference value is exceeded, and if it is exceeded, it is determined that there is an error or problem, and the result of the judgment can be provided to a manager set in advance. Also, here, the machine learning-based learning may be a neural network or a pattern classification learning algorithm using at least one known CNN, RNN, SVM, or the like.

The database 150 may collect, store, and manage member information, video information, video review information, statistics, and the like.

In this specification, a 'terminal' may be a wireless communication device that guarantees portability and mobility, and may be, for example, any type of handheld-based wireless communication device such as a smart phone, a tablet PC, or a laptop computer. In addition, the 'terminal' may also be a wired communication device such as a PC capable of accessing other terminals or servers through a communication network. In addition, a communication network refers to a connection structure capable of exchanging information between nodes such as terminals and servers, such as a local area network (LAN), a wide area network (WAN), and the Internet (WWW : World Wide Web), wired and wireless data communications network, telephone network, and wired and wireless television communications network.

Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC: Visible Light Communication), LiFi, and the like, but are not limited thereto.

100; Service provision server
110; Ministry of Communications
120; Management
130; display control unit
140; Department of Statistics
150; database
200; User terminal
300; communications network
V; video information department
T; time display
I; section information department

Claims (7)

When the user inputs a video file to be checked and reviewed, the video review program executed on the screen of the user terminal analyzes the video file and extracts the video data and audio data;
The video review program extracts at least one video still picture for each section from the extracted video data in units of a review time interval input by a user and outputs the still picture to the screen of the user terminal;
generating, by the video review program, subtitle information and volume information for each section corresponding to the video still screen for each section separately from the extracted audio data;
Classifying the video still screen of the corresponding part and outputting the video still screen for each section, subtitle information, and volume information to the screen of the user terminal;
the video review program extracting audio data through analysis of a video file to be reviewed;
The video review program generates PCM data for each section according to the review time interval and subtitle output time input by the user, and transmits the audio data to a service providing server;
the service providing server inputting the received PCM data to the STT engine, generating an STT conversion result, and transmitting the result to a user terminal where the video review program is executed;
The video review program further includes displaying and providing subtitle information for each section corresponding to a video review time interval by dividing and outputting the STT conversion result by section,
When a request is made to the STT engine for STT conversion of audio extracted from a video to be reviewed in the user terminal, the language information to be converted is transmitted together by allowing a desired country language to be selected, and the STT engine converts the STT converted into the corresponding language Provide a conversion result, but provide a plurality of conversion results upon request for a plurality of languages,
Errors in the subtitle information for each section identified by the user terminal are corrected through user modification and conversion to check the corresponding audio information for each section, and the corresponding section is matched with the caption information and the video still screen for each section. Subtitle sync is adjusted by checking whether it corresponds to the subtitle of
When the video review program is executed in a web-based form on the user terminal, log-in authentication is performed on the service providing server, and a blockchain network is established in conjunction with a plurality of blockchain servers, through the established internal blockchain network. Generates public and private keys, converts them into hash values, distributes and stores them, performs user login authentication based on the distributedly stored public key and user's personal information, and performs user login authentication together with the public key in the plurality of user terminals. Information is received and user certificates including a hash value for the user information are generated, and each user certificate is stored using a Merkle tree structure.
Video review method including. According to claim 1,
The video review program
a video information unit for displaying a video playback screen, a menu related to video playback, and information on a video file so as to display information about the video;
When reviewing the video displayed in the video information unit, a time display unit for displaying a corresponding video review time interval to receive an input and displaying a caption output time to confirm where the currently output caption corresponds to a playback time section of the video;
When reviewing the video displayed in the video information unit, including a section information unit for displaying section information including at least one video still screen, subtitle information, and volume information for each section for the video review time interval input from the time display unit. Featured video review method. delete A video review program stored in a computer-readable storage medium that performs the video review method of claim 1 or 2. In the video review system using the video review method of claim 1 or 2,
An STT engine that is connected to a user terminal where the video review program is executed through a communication network and converts the subtitle text to the audio when the video review program executed in the user terminal requests subtitle text conversion of the audio extracted from the video to be reviewed. A service providing server having
Video review system that includes. According to claim 5,
The service providing server
A communication unit having at least one communication protocol for communicating with a user terminal through a communication network and performing data communication of information necessary to provide a video review service;
a management unit having an STT engine to perform audio-caption text conversion on the video to be reviewed, and transmitting the STT conversion result to the user terminal through the communication unit;
Performs a control function for the user interface implemented in the video review program, plays a video from a still video screen or plays volume information according to a user input operation through the video review program, and performs a user input operation for subtitle information. video review system including a display control unit that provides an interface so that the text of the corresponding subtitle information can be directly re-entered and corrected. According to claim 6,
The service providing server
As the video review program is provided, the video review history is collected, and the information is managed so that the information can be viewed upon user request. Statistical learning that further includes user evaluation or satisfaction, can perform machine learning-based learning using the collected video review history, and determines the results of problems or errors in the video review program as a learning result and provides it to the manager A video review system that includes more wealth.

Images