ITBA20100024A1 - "SYSTEM FOR MONITORING, RESEARCH, REVIEW, INTEGRATION AND AUDIO / VIDEO / TEXT EDITING BASED ON TRANSLATION OF SPEAKING CONTENT IN MULTIMEDIA SEQUENCES BY SPEECH RECOGNITION AND SPEECH TO TEXT SYSTEMS". - Google Patents

Description

Description of the Patent for Industrial Invention entitled:

â € œSystem for monitoring, research, review, integration and audio / video / text editing based on the transcription of speech contained in multimedia sequences using speech recognition and speech to text indexing systemsâ €

The present invention is part of the research systems sector, implemented by computer means and, more particularly, in the methods which allow to monitor, search, examine the content of multimedia sequences.

The invention concerns a structured set of advanced technologies such as automatic speech transcription, text indexing methodologies, search engines as well as web technologies and services for accessing and managing the search for information at internal audio-video content from various sources such as TV, radio, web, digital or digitized archives, aimed at the searchability and integration of multimedia content with the rest of the content on the web.

The main known technique used today for research on large quantities of multimedia contents is based on the categorization of said contents on the basis of comments associated with the files. The association can take place either through the use of qualified personnel who provide for the categorization, or through comments entered directly by those who use the system, as a contributor of contributions, as is the case, for example, with YouTube®.

The resulting workload will therefore be proportional to the quantity of videos reviewed.

Furthermore, the current technique provides a search result depending on the quality and descriptive accuracy of the contents inserted to categorize the files. The purpose of the present invention is to provide a decidedly effective system for searching within audio-video documents and capable of archiving large amounts of recordings while maintaining an accurate index of contents.

A further purpose is to provide a suitable system for generating structured indexes in order to be able to quickly and punctually find the document most pertinent to the topic chosen by the applicant.

The present invention, for the aforementioned purposes, provides for the integration of both an automatic voice recognition (ASR) based on SILVCSR (Speaker Independent Larg-Vocabulary Continuous Speeck Recognition) engine and a latest generation indexer and search engine, allowing the system to search directly on the â € œspokenâ € contained in the documents.

These and further advantages are achieved by the system for monitoring, research, review, integration and audio / video / text editing based on the transcription of speech contained in multimedia sequences using speech recognition and speech indexing systems. to text, referred to in the present invention, described with the help of the attached drawing table which illustrates the following figure:

fig. 1 a block diagram of the system.

As shown in figure 1, the system is structured in such a way as to receive multimedia documents 1 as input, to produce a transcription of speech 2 of the audio-video content - containing references to the exact millisecond in which each single word is pronounced â € “and submit everything to the indexing process 4,5 and 6. It will then be the task of the web interface 7 to provide the end user with the modalities for a simple and immediate consultation, as well as a set of resources additional.

Furthermore, an automatic feedback system 8 and 9 is provided, controlled by the user, which allows a dynamic and continuous fine-tuning of the service offered and of the structure of the system itself.

Blocks 3a and 3b highlight the need to store data, from various sources and encodings, in a uniform format as required for the efficiency of the system and to extract the still image of any video file for precise references to meaningful sentences and / or relevant segments of the original audio file.

Block 4, on the other hand, highlights the possibility of the intervention of a human operator for the correction of transcription errors using a special software tool and guidelines defined on the basis of efficiency criteria and continuously updated on the basis of feedback.

As claimed, the system object of the invention has the following basic characteristics:

- taking charge 2 of the material, by the system, and sorting it to the automatic transcription system and, possibly and subsequently, to a revision system that makes use of specialized personnel 4;

- production and storage of the recoded multimedia files 3a, or thumbnails 3b, only in the case of videos, extracted on the basis of the subdivision into sentences / segments obtained from the automatic transcription and possible revision 4;

- processing and indexing of the text for the search engine and its availability for consultation 6;

- a web interface will take care, finally, of the modalities with which the data are made usable by the end user and of the additional services 7;

The correction and transcription service by operator through a specialized software for correction, which integrates and completes the raw automatic transcriptions, helps to produce perfect transcriptions, or to verify the correct transcription of proper names of person, place and organization, and to manage any publication errors 7.

An example of an application is specialized monitoring of television, radio and web audio video content.

It is therefore a precise and powerful application, capable of generating virtuous savings of time and resources, especially where the amount of audio video content to be treated is vast and articulated.

Claims (9)

CLAIMS 1. â € œSystem for monitoring, research, review, integration and audio / video / text editing based on the transcription of speech contained in multimedia sequences using speech recognition and speech to text indexing systemsâ € characterized by the integration of both an automatic voice recognition (ASR) engine based on SILVCSR (Speaker Independent Large-Vocabulary Continuous Speech Recognition) engine and a latest generation indexer and search engine, allowing the system to search directly on the â € Spoken contained in documents. 2. System for the acquisition of audio-video material from different sources, characterized by a sequence of instructions stored in an optical magnetic disk-type storage medium consisting of: - taking charge 2 of the material, by the system, and its sorting to the automatic transcription system and, eventually and subsequently, to a revision system that makes use of specialized personnel 4. - production and storage of the recoded multimedia files 3a, or thumbnails 3b, only in the case of videos, extracted on the basis of the subdivision into sentences / segments obtained from the automatic transcription and possible revision 4. - processing and indexing of the text for the search engine and its availability for consultation 6; - a web interface will take care, finally, both of the modalities with which the data are made usable by the end user and of the additional services 7. 3. System referred to in claims 1 and 2 characterized by the fact that whenever a keyword or a topic of interest is dealt with by a television or radio broadcaster, an alert is sent automatically and promptly via e-mail or sms to the computer or telephone of the service user. 4. System referred to in the preceding claims, characterized by the fact that the video clip containing the part of the film, news report, interview or dossier transmitted by the monitored television, radio or web broadcasters, corresponding to the keyword or topic of interest indicated à ̈ made immediately accessible to the user of the service. 5. System as per the preceding claims characterized by the fact that the video clip is published on a web interface thus allowing its download. 6. System referred to in the preceding claims characterized by the fact that the cutout is supplied complete with its integral transcription in an interactive and printable version, point & click on the textual track with immediate redirection to the corresponding point. 7. System referred to in the preceding claims characterized by the fact that a search in the archive is allowed, assisted by the system itself, and carried out by means of a latest generation search engine, the latter being able to be carried out according to traditional canons, or by crossing information such as date, issuer, topic. 8. System referred to in the previous claims characterized by the fact that each extract extracted is classified on the basis of predefined categories that can be configured according to criteria established by the user of the service and, by the fact that the system itself, through the search engine and working on indexed texts, will increase the category index thus generating an automatic classification of the examined resources. Moreover, the system, thanks to the ability of the search engine to structure originally unstructured content, is able to use filters created ad hoc according to the customer's needs, effectively restricting the search field on the available audio / video contents. 9. System referred to in the preceding claims characterized by the fact that statistical data are generated on the basis of the occurrences of words and contents, the data thus structured can be consulted in the form most suitable for the research to be conducted.