CN104144162A - A method and system for realizing intelligent voice linkage shouting - Google Patents

Abstract

The invention discloses a method and system for realizing intelligent voice linkage shouting. The system mainly includes: an audio device, a video device, a storage server, a media server, a management server, and main hardware devices of a client; the software adopts a C/S architecture; the method It mainly includes: intelligent voice linkage shouting can trigger conditional voice automatic shouting through the video device; the shouting function can be turned on through the client software and the front-end video device and voice device can be used for on-site voice shouting. In theory, the server can support unlimited client access, which is mainly determined by bandwidth and computer performance. By adopting the present invention, it is possible to realize shouting alarms and real-time shouting exchanges between objects (equipment) and people in an unattended and real-time operation environment, thereby improving efficiency and ensuring safety.

Description

A method and system for realizing intelligent voice linkage shouting

technical field

The invention relates to multimedia technology and communication technology, in particular to a method and system for realizing intelligent voice linkage calling.

the

Background technique

With the continuous development of multimedia technology and communication technology, the application of on-site shouting is widely used. The main way to realize it is to use the intercom function integrated with wireless communication and voice technology. See Figure 1 for the flowchart of voice shouting in the prior art. The process includes the following steps: step 101, open or close the shouting function between devices, step 102, one-way shouting between users; there are voice intercoms that integrate video and audio through wired or wireless communication; there are also conference system-based development voice system. The existing technologies are all developed for the purpose of smoother voice information and more diverse information. They are all direct communication between people, based on point-to-point or point-to-multipoint information interaction between people in real time.

In practical applications, it is necessary for people or objects (equipment) to make voice calls to each other or according to judgment conditions, select different call content according to different conditions, and use intelligent voice linkage call for management as the main purpose. However, there is no corresponding technology in the prior art, so the present invention provides a method for intelligent voice calling. The initiation of voice calling can be triggered by a front-end intelligent video device or can be triggered by a back-end client. The present invention is developed based on the TCP/IP protocol , improve the intelligence, flexibility, and efficiency of voice announcements, realize the splitting of voice and video, and realize the functions of front-end intelligent voice linkage announcements and client real-time voice announcements.

the

Contents of the invention

Aiming at the above-mentioned defects of the prior art, the present invention provides a method and system for realizing intelligent voice linkage shouting that solves the problem of intelligent voice shouting in an unattended environment and is compatible with the real-time voice shouting function during manned operations.

The present invention provides a method and system for realizing intelligent voice linkage shouting. Since the system and method have the same specific technical features: that is, it can be triggered by the front-end intelligent video device or the back-end client can start the shouting function, so they belong to the same inventive concept and can be Filed as one application.

In order to achieve the above-mentioned purpose of the invention, the technical solution of the present invention is: a method for realizing intelligent voice-linked shouting, which is characterized in that the video device recognition method is used to automatically link the voice shouting of the audio device and the real-time shouting of the client, and the method includes the following steps:

A. The front-end video device determines whether to automatically link voice calls according to the type of event analyzed by the intelligent analysis module; when an intelligent voice linkage call is required, the audio channel is turned on;

or B. The client terminal enables the real-time calling function, and reaches the audio and video device and the scene through the media server for real-time calling;

The concrete process of described step A is as follows:

A00, the video device checks the equipment and the environment, uses the intelligent analysis module to analyze the scene and takes the call linkage function;

A01, the video device allocates audio channel resources for shouting, and prepares audio shouting content;

A02, send the shouting content to the audio device, and shout the alarm to the scene;

The specific process in the described step B is as follows:

B00, the client user opens the calling function, at this time the management server checks the status and permissions of the client;

B01, the management server assigns and authenticates the client user, and the media server assigns channel resources to the video device;

B02, the media server forwards the content of the client's shouting to the video device, and the video device opens the audio channel to send the shouting content; at the same time, the front-end on-site shouting content is collected by the audio device and uploaded to the media server and then forwarded to the client user.

The aforementioned method for implementing intelligent voice linkage shouting, in step A00, when the intelligent linkage voice shouting action, the audio channel is not opened when the analyzed video scene does not meet the conditions for intelligent shouting, and the intelligent analysis module logically associates the video scene and the shouting content , using the event method, judging the shouting method and shouting duration by the event type, and analyzing the event level according to the definition, the shouting alarm prompt, shouting alarm processing, and shouting information are notified to the management personnel for processing; in the step A01, the shouting content is in *.WAM The file format is recorded in the SD card of the video device.

In the aforementioned method for implementing intelligent voice linkage shouting, in step B01, when there are multiple clients performing intelligent shouting operations on the same video device and audio device, at this time, they are treated differently according to the level of authority. sequential selection.

In the aforementioned method for implementing intelligent voice linkage calling, in step B01, after the management server judges whether to enable real-time calling according to the authority, the management server inquires whether the access ID and IP address of the video device are correct, and then inquires about the operating status and communication status of the video device , when everything is normal, the media server allocates audio channel resources to the video device and forwards the client user audio data to the video device.

In the aforementioned method for implementing intelligent voice-linked shouting, after the client shouts out, the client needs to manually turn off the shouting function, and the media server will notify the video device to close the audio channel to release resources.

In the aforementioned method for implementing intelligent voice linkage shouting, the communication link is disconnected during the intercom, and the management server sends heartbeat data to the front-end video device every 5 seconds. If no video device data is received within 5 seconds , it is considered that the device communication is faulty, and this information will be notified to the client according to the client ID. If it is not normal within 5 heartbeat cycles, the media server will be notified to close the audio channel resource.

An intelligent voice linkage calling system is characterized in that it consists of the following parts:

The audio device includes an audio playback unit and an audio acquisition unit, which are respectively used to send the decoded audio content to the scene, collect and transmit the content of the scene to the video device for encoding;

The video device is used to encode the audio after collecting and then transmit it to the media server and storage server, provide decoding function for the audio playback content, and pre-store the audio file to provide content for the intelligent voice call;

The client is used to turn on or off the voice call in real time. After the real-time voice call is processed by the media server and the management server, the real-time voice call is provided to the audio and video devices, and it is also used to receive the on-site voice call content of the audio and video device forwarded by the media server;

Storage server, which provides storage space for the shoutout content and process;

The media server is used to allocate audio channel resources and forward audio and video data to the video devices connected to the front end;

The management server centrally manages the client, storage server, media server, video device device information, device status, and user rights.

In the aforementioned intelligent voice-linked shouting system, the video device is equipped with an intelligent analysis module, and the intelligent analysis module logically associates the video scene with the shouting content, adopts the event method, judges the shouting method and the shouting duration by the event type, and analyzes the event level according to the definition. Call out alarm prompts, call out to alarm processing, and call out information to notify management personnel for processing.

Aforesaid a kind of intelligent speech linkage calling system, video device has audio codec unit, audio storage unit, audio transmission unit; Described audio codec unit provides G.711 standard audio frequency for shouting audio frequency; Described audio storage unit is An SD card for recording the audio content of the call; the audio sending unit sends the information collected by the audio device to the media server and forwards it to the client. the

The invention provides a method for realizing intelligent voice linkage shouting, reflecting the flexibility and intelligence of shouting; the front-end video device can intelligently link the shouting device according to the video analysis results to provide flexible shouting content at the same time, and realize things and things, things and things when no one is on duty. Information transmission between people; when someone is on duty, the client can start a real-time audio call to realize the information transmission between people.

the

Description of drawings

The flow chart of voice shouting in the prior art of Fig. 1;

Fig. 2 is the basic flow chart of the intelligent linkage voice shouting of the front-end video device according to the embodiment of the present invention;

Fig. 3 is the basic flow chart of the client starting the real-time voice calling in the embodiment of the present invention;

Fig. 4 is a detailed flow chart of the intelligent linkage voice shouting of the front-end video device according to the embodiment of the present invention;

Fig. 5 is a detailed flow chart of the client enabling real-time voice calling in the embodiment of the present invention;

Fig. 6 is a system structure diagram of an embodiment of the present invention;

Fig. 7 is a structural diagram of a front-end device according to an embodiment of the present invention.

the

specific implementation plan

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

The intelligent voice call provided by the embodiment of the present invention is mainly composed of the intelligent linkage voice call of the front-end video device and the real-time voice call of the client. Referring to Fig. 2 and Fig. 3, Fig. 2 and Fig. 3 are basic flowcharts of the embodiment of the present invention. As shown in Figure 2, the front-end video device intelligently links voice calls, and the process includes the following steps:

Step 201, the video device records and saves the shouting content, and sets the video device intelligent analysis to trigger the voice shouting condition;

What needs to be explained here is that the pre-recorded audio content in step 201 can be thousands of chanting audios, and the on-site chanting content can be pre-recorded and stored in the SD card according to different requirements, and the pre-stored audio format is *.wam.

Step 202, the front-end video device starts the voice call according to the video intelligent analysis result;

The video device is equipped with an intelligent analysis module to logically correlate the video scene and the content of the call, adopts the event method, and judges the call method and call duration by the event type.

What needs to be explained here is that, according to the definition of the event method, the analysis of the event level will use the shouting alarm prompt, shouting alarm processing, and shouting information to notify the management personnel to deal with it; the definition of shouting duration, according to the intelligent analysis results, adopts the shouting duration as 5 consecutive times.

Step 203, the front-end audio device pushes the shouting content according to the intelligent video device, and sends out a shouting alarm to the surroundings;

As shown in Figure 3, the client starts real-time voice calls, and the process includes the following steps:

Step 301, the client enables the real-time voice calling function;

Step 302, the media server forwards the content of the voice call to the front-end video device, and the video device pushes the content of the call to the audio device. the

Step 303, the front-end audio device plays the shouting content, collects live audio in real time and transmits it to the client;

So far, the basic flowchart provided by the embodiment of the present invention has been realized through the above steps.

The method of the present invention has been basically described above. In order to make the method of the present invention clearer, the method of the present invention will be described in detail below.

Referring to Fig. 4 and Fig. 5, it is a detailed flowchart of the present invention. As shown in Figure 4, the front-end video device intelligently links voice calls, and the process may include the following steps:

In step 401, the video device sends an intelligent voice call to the audio device when the intelligent analysis condition is triggered.

The video device uses the intelligent analysis module to analyze the video scene. After the analysis results, it is decided whether to use linked voice calls. If it is necessary to use linked calls, it will automatically turn on the call switch and prepare to call out to the audio device.

Step 402, do not open the audio interface channel when the analysis condition does not meet the intelligent call condition.

Step 403, when the analysis condition meets the condition of intelligent utterance, link the *.WAM audio content of the video device, pre-record multiple audio source files according to different utterance contents, and allocate the audio content that needs to be uttered.

Step 404, enable the calling function, and turn on the audio channel switch.

It should be noted here that the real-time audio call adopts the G.711 codec format, which is separated from the video information.

Step 405, calling out to the scene, and sending the audio content of the calling out.

It should be noted here that the audio device and the video device only perform one-way shouting at this time, and collect the sound of the scene state. After the call is complete, the video device closes the audio channel.

As shown in Figure 5, the client starts real-time voice calls, and the process may include the following steps:

Step 501 , the client user sends a real-time calling function to the front-end video device.

What needs to be explained here is that when there are multiple clients performing intelligent calling operations on the same video device and audio device, they will be treated differently according to the level of different permissions; The user counts for 60 seconds, and the rotation mode goes to the next client user.

Step 502 , the management server assigns and authenticates the client user's rights, and if the client user passes the authentication and has the right to open the callout, he will click "start callout" successfully.

Step 503 , after receiving the command to enable the calling function, the management server checks whether the access ID and IP address of the video device are standardized.

What needs to be explained here is that the management server performs standardized access management on the video device, the client, and the media server, and at the same time detects the running status.

Step 504, the management server detects the communication status and running status of the video device.

Step 505, after the management server detects that the status is normal, the media server allocates channel resources of the video device, and at the same time allocates an audio memory buffer, and forwards the audio to the video device.

Step 506, the video device opens the audio port of the calling function, and decodes the audio data of the client; at the same time, encodes the sound collected by the audio device and uploads it to the media server.

Step 507, the audio device sends out the real-time shouting content of the client operator, and at the same time collects the on-site sound.

What needs to be explained here is that after the on-site communication with the client through the audio device is completed, the client needs to manually turn off the function of calling. At this time, the media server will notify the video device to close the audio channel to release resources.

In step 508, the shouting content is formed into an audio file, which is encoded by the video device and uploaded to the media server while forwarding a copy to the storage server.

So far, the flow provided by the implementation of the present invention is ended.

There is also a special case in the above, when the communication link is disconnected during the intercom, the management server will send heartbeat data to the front-end video device every 5 seconds, if no video device data is received within 5 seconds, it will be considered If the device fails to communicate, it will notify the client of this information according to the client ID. If it fails to work within 5 heartbeat cycles, it will notify the media server to close the audio channel resource. The calling function of the client is the same as turning off.

The above is the description of the method for implementing the call of the present invention, and the system and equipment provided by the present invention will be described below in conjunction with specific embodiments.

Referring to FIG. 6, FIG. 6 is a structural diagram of the system of the present invention. As shown in FIG.

The two-way data communication between the audio device 601 and the video device 602, the audio device 601 provides live sound collection data for the video device 602, and the video device 602 provides the audio device 601 with a shouting sound source or a client real-time shouting sound source.

The video device 602 and the management server 603 are one-way data communication. When the video device is initialized, it registers with the management server, and sends the device type ID and IP address to confirm with the management server. If it is correct, it will normally register with the management server, otherwise the management server will not proceed. deal with. After normal, send heartbeat data to the management server every 5 seconds, and send status data every 5 seconds.

The storage server 604 performs one-way data communication with the management server 603. When the storage server is initialized, it registers with the management server, and sends the device type ID and IP address to confirm with the management server. If it is correct, it will normally register with the management server, otherwise the management server will not deal with. After normal, send heartbeat data to the management server every 5 seconds, and send status data every 5 seconds.

The storage server 604 communicates with the media server 605 bidirectionally, and the storage server only receives the audio data forwarded by the media server.

The media server 605 performs one-way data communication with the management server 603. When the media server is initialized, it registers with the management server, and sends the device type ID and IP address to confirm with the management server. If it is correct, it will normally register with the management server, otherwise the management server will not deal with. After normal, send heartbeat data to the management server every 5 seconds, and send status data every 5 seconds.

The media server 605 communicates with the client 606 in two-way data communication, the client accesses the media server data, and the media server also forwards the client audio data.

In the embodiment of the present invention, the front-end equipment audio device and the video device have multiple structural forms in specific implementation, and one of the structures shown in FIG. 7 below is described as an example. Referring to FIG. 7, FIG. 7 provides a structural diagram of a front-end device according to an embodiment of the present invention. As shown in Figure 7, the audio device includes an audio playback unit 701 and an audio acquisition unit 702; the video device includes an audio sending unit 703, an audio storage unit 704, and an audio encoding/decoding unit 705

Among them, the audio playing unit 701 plays the shouting content sent by the video device, and plays the shouting content of the client. The audio collection unit 702 collects and uploads live sounds. The audio sending unit 703 is used to package and send the audio data to the media server through the TCP/IP protocol by the audio sending unit of the video device.

In the audio storage unit 704, the video device stores the data of the audio collection unit, directly stores it in an SD card, or forwards it to the storage server for storage by the media server.

The audio codec unit 705 mainly performs pre-processing of audio transmission, and codes/decodes into G.711 format.

It can be seen from the above technical solutions that the present invention provides a method for implementing intelligent voice linkage shouting, which reflects the flexibility and intelligence of shouting; the front-end video device can intelligently link the shouting device according to the video analysis results while providing flexible shouting content to realize unattended Information transmission between things and things, things and people; when someone is on duty, the real-time audio call can be turned on by the client to realize the information transmission between people.

The invention can be summarized by other specific forms that do not violate the spirit and main features of the present invention. Therefore, the above-mentioned embodiments of the present invention can only be considered as illustrations of the present invention and cannot limit the present invention. They are equivalent to the claims of the present invention. Any changes within the meaning and scope of the claims should be considered to be included in the scope of the claims.

Claims (9)

1. the intelligent sound interlock implementation method of propagandaing directed to communicate, is characterized in that, utilizes video-unit recognition methods automatic linkage audio devices voice to propaganda directed to communicate and client is propagandaed directed to communicate in real time, and the method comprises the following steps:

A, front end video-unit are analyzed event type according to intelligent analysis module and are determined whether automatic linkage voice are propagandaed directed to communicate; While needing intelligent sound interlock to propaganda directed to communicate, open voice-grade channel;

Or B, client open the function of propagandaing directed to communicate in real time, arrive Voice & Video device and propaganda directed to communicate in real time in scene through media server;

The detailed process of described steps A is as follows:

A00, video-unit checks equipment, environment, utilizes intelligent analysis module to analyze scene and the interlock merit of taking to propaganda directed to communicate is done;

A01, video-unit distributes the voice-grade channel resource of propagandaing directed to communicate, and prepares the audio frequency content of propagandaing directed to communicate;

A02, sends and propagandas directed to communicate content to audio devices, to the scene alarm of propagandaing directed to communicate;

Detailed process in described step B is as follows:

B00, client user opens propaganda directed to communicate function, now management server viewing client-side state, authority;

B01, management server carries out right assignment and certification to client user, and media server is that video-unit distributes channel resource;

B02, the client content of propagandaing directed to communicate is forwarded to video-unit by media server, opens voice-grade channel send the content of propagandaing directed to communicate by video-unit; Propaganda directed to communicate after content is gathered by audio devices and be uploaded to media server and then be forwarded to client user in front end scene simultaneously.

2. a kind of intelligent sound interlock according to claim 1 implementation method of propagandaing directed to communicate, it is characterized in that, in steps A 00, when action propagandaed directed to communicate in intelligent linkage voice, video scene does not conform with and when intelligence is propagandaed directed to communicate condition, does not open voice-grade channel by analysis, intelligent analysis module is carried out logic association to video scene and the content of propagandaing directed to communicate, adopt event mode, by event type judgement propaganda directed to communicate mode and the duration of propagandaing directed to communicate, according to defined analysis event class by employing propaganda directed to communicate alarm prompt, propaganda directed to communicate report to the police process, the information of propagandaing directed to communicate notifies administrative staff to process; In described steps A 01, the content of propagandaing directed to communicate is with in * .WAM file format typing video-unit SD card.

3. a kind of intelligent sound interlock according to claim 1 implementation method of propagandaing directed to communicate, it is characterized in that: in step B01, when existing multi-client to carry out the intelligence operation of propagandaing directed to communicate to same video-unit and audio devices, now treat with a certain discrimination according to authority height, if authority is identical according to the selective sequential of First come first served.

4. a kind of intelligent sound interlock according to claim 1 implementation method of propagandaing directed to communicate, it is characterized in that, in step B01, after management server judges whether that according to authority unlatching is propagandaed directed to communicate in real time, whether management server queries video-unit access mark and IP address be correct, inquire about video-unit running status and communications status, when all are normal, media server distributes voice-grade channel resource and forwards client user's voice data to video-unit to video-unit again.

5. a kind of intelligent sound interlock according to claim 1 implementation method of propagandaing directed to communicate, is characterized in that, after client has been propagandaed directed to communicate, client needs the manual-lock function of propagandaing directed to communicate, and now media server will inform that video-unit cuts out voice-grade channel releasing resource.

6. a kind of intelligent sound interlock according to claim 1 implementation method of propagandaing directed to communicate, it is characterized in that, in the process of intercommunication, communication link disconnects, management server carries out 1 heartbeat data with front end video-unit in every 5 seconds and sends, if while not receiving video-unit data in 5 seconds, think devices communicating fault, will be according to client id by this information announcement client, also abnormal in 5 heart beat cycles, notice media server is closed voice-grade channel resource.

7. the intelligent sound interlock system of propagandaing directed to communicate, is characterized in that, is made up of following part:

Audio devices, comprises audio playing unit and audio collection unit, is respectively used to audio content after decoding to send the content of propagandaing directed to communicate, the scene content acquisition of propagandaing directed to communicate is sent to video-unit coding scene;

Video-unit, is resent to media server and storage server for encoding after audio collection, for audio frequency play content provides decoding function, the audio file that prestores is propagandaed directed to communicate content is provided for intelligent sound;

Client, propaganda directed to communicate for opening or close voice in real time, real-time voice is propagandaed directed to communicate provides real-time voice to propaganda directed to communicate through media server and the backward sound of management servers process, video-unit, the on-the-spot voice of audio-video equipment that also forward for the receiving media server content of propagandaing directed to communicate;

Storage server, in order to propaganda directed to communicate, content and process provide memory space;

Media server, forwards the propaganda directed to communicate distribution of voice-grade channel resource and sound, video data of the video-unit of front end access;

Management server, centralized management client, storage server, media server, video-unit facility information, equipment state, user right.

8. a kind of intelligent sound interlock according to claim 7 system of propagandaing directed to communicate, it is characterized in that: video-unit arranges intelligent analysis module, intelligent analysis module is carried out logic association to video scene and the content of propagandaing directed to communicate, adopt event mode, by event type judgement propaganda directed to communicate mode and the duration of propagandaing directed to communicate, according to defined analysis event class by employing propaganda directed to communicate alarm prompt, propaganda directed to communicate report to the police process, the information of propagandaing directed to communicate notifies administrative staff to process.

9. a kind of intelligent sound interlock according to claim 7 system of propagandaing directed to communicate, is characterized in that: video-unit has audio coding decoding unit, Audio storage unit, audio frequency transmitting element; Described audio coding decoding unit provides G.711 standard audio for the audio frequency of propagandaing directed to communicate; Described Audio storage unit is the SD card of the audio content of can typing propagandaing directed to communicate; Described audio frequency transmitting element, is sent to audio devices Information Monitoring media server and is transmitted to client.