JPH04302561A - Multi-media communication system - Google Patents

Abstract

PURPOSE:To execute the communication between each terminal of different kinds of media. CONSTITUTION:In the system which can execute a communication at least between each communication terminal of the same kind of media by connecting various media such as a voice terminal, a text terminal for executing a data communication, an image communication terminal for giving and receiving image information, etc., it is constituted of a processor 6 containing a data converting means 64 for receiving a sound signal, recognizing it as a voice, analyzing it, converting it to text data (text data) and outputting it, image recognizing/converting means 72, 76 for receiving an image information signal, extracting character information, recognizing a character pattern, converting it to the text data corresponding to a result of recognition and outputting it, a voice synthesizing means 67 for converting these output text data to a sound signal and outputting it, and image conversion output means 73, 78 for converting the output text data to image information and outputting it, and an exchange means for confirming the media of the other party terminal at the time of communication, giving an output of the transmitting terminal to the processor, and also, controlling the processor in order to convert it to a signal corresponding to the media of a receiving terminal and sending the converted output to the receiving terminal.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia communication system capable of carrying out telephone calls and data communications such as voice communication, text data transmission, and handwritten character transmission.

0002

BACKGROUND OF THE INVENTION There are multimedia communication systems that allow mutual voice communication and mutual communication of text and handwritten characters. In this system, voice can be communicated with a telephone terminal, text can be communicated with a data terminal such as a personal computer, and handwritten information can be communicated with a facsimile machine.

FIG. 8 shows an example of a multimedia communication system. The figure shows an example of a communication system centered on a PBX (private branch exchange) 1, which includes data terminal devices such as telephones 2a, 2b, personal computers 3a, 3b, facsimile machines 4a, 4b, tablets 5a, 5b, etc. A terminal device, etc. is connected. In this system, telephone calls and communications between these communication terminals are carried out as indicated by broken lines in the figure: the telephone 2a is connected to the telephone 2b, and the facsimile device 3
a is the facsimile machine 3b, and computer 4a is the computer 4
This is implemented between terminals with the same attributes, such as b.

However, in the field of data communications, protocol conversion, etc. within the network is supported as a VAN service. Also, as one of the banking services, there is a voice response for deposit balance, but although the source is data, there are various forms of information such as voice output to the terminal. In some cases, it may be changed.

[0005] However, in normal telephone calls and communication services,
As mentioned above, it is a call or communication between people with the same attributes,
For example, if the other party's terminal is a facsimile machine, the sender must create a document, convert it into an image signal using the facsimile machine, and send it.

[0006]

SUMMARY OF THE INVENTION In a multimedia communication system, it is possible to perform mutual voice communication and mutual communication using text and handwritten characters. However, communications can only be made between devices with the same attributes, such as telephones for voice communication and facsimile machines for written communication. This is because audio terminals can only convert audio signals and cannot handle data or image signals.
On the other hand, this is because facsimile machines cannot handle voice signals or data, and the signals that they can handle are different.

On the other hand, in a PBX (private branch exchange),
With the advancement of functionality, convenient functions such as an answering machine have been added, and even if the phone is an extension phone, messages from the caller to the extension can be recorded on the voice mail device inside the PBX, and later sent to the phone. The user of the corresponding extension telephone can play and listen to this voice message and learn the message.

[0008] However, in this case, when there are many incoming calls and many messages, it takes a considerable amount of time to listen to all of them. It is extremely difficult to check incoming messages while the person is away, such as having to repeat the process over and over again.

[0009] Also, not all of the messages are important, and it is desirable to be able to confirm the entirety more easily. Furthermore, if terminals using different media cannot communicate with each other at all, operations may end up being a waste of effort, such as when a call is unknowingly connected.

[0010]Also, despite the development of communication networks, it has not become easy for deaf and able-bodied people to communicate with each other. It would be helpful for both parties to communicate if a hearing-impaired person could use text input/output, but even though various media are connected on the same network, the media are different It is such a shame that it cannot be used in this way.

[0011] Therefore, the purpose of this invention is to
Media conversion allows for healthy people (audio input/output) and hearing-impaired people (
This makes it possible to make calls with text input/output), to enable communication between different media, and to shorten the playback time by converting answering machines into text data and viewing them at a glance, making it easier to check the contents. An object of the present invention is to provide a multimedia communication system that can further improve services.

0012

[Means for Solving the Problems] In order to achieve the above object, the present invention is constructed as follows. That is, in a multimedia communication system that connects various media such as voice terminals for making phone calls, text terminals for data communication, and image communication terminals for sending and receiving image information, and enables communication between communication terminals using at least the same type of media. , a text data conversion means that receives a voice input signal, performs voice recognition from it, analyzes it, converts it into text data, and outputs it; and a text data conversion means that receives an image information signal, extracts character information from it, recognizes the character pattern, and outputs the text data. an image recognition conversion means for converting and outputting text data corresponding to recognition results; a speech synthesis means for converting and outputting the output text data into speech; and a voice synthesis means for converting the output text data into image information. A signal processing device comprising an image conversion output means for outputting an image, and a signal processing device that checks the media of the terminal device to be the communication partner at the start of communication, gives the output of the terminal device to be transmitted to the signal processing device, and outputs the output of the terminal device to be received. An exchange in which the signal processing device is controlled to convert it into a signal compatible with the media, and the output of the transmitting terminal device is given to the signal processing device, and the converted and output signal is sent to the receiving terminal device. means.

[0013]

[Operation] In this device having such a configuration, at the start of communication, the exchange means checks the media of the terminal device to be communicated with, provides the output of the terminal device to be transmitted to the signal processing device, and outputs the output of the terminal device to be received. It controls the signal processing device to convert it into a signal compatible with the media, and provides the output of the transmitting terminal device to the signal processing device. The signal processing device converts the output of the transmitting terminal device into text data, converts it into a signal format compatible with the media of the receiving terminal device, and outputs it to the exchange means. The converted and output signal is sent to the receiving target terminal device.

[0014] In this way, the output from the transmitting terminal is first converted into text data, the signal form of this text data is converted in accordance with the media of the receiving terminal, and then the signal is given to the receiving terminal. be. Specifically, for example, a signal processing device with a voice recognition synthesis function or a handwritten character recognition and encoding function is added to the PBX, and voice and handwritten characters are converted into text data by voice recognition and character pattern recognition. This text data is sent to voice terminals by converting it into an audio signal using speech synthesis, and is sent to facsimiles and drawing communication terminals by converting it into an image signal, so the media are different. It becomes possible to make calls and communicate between communication terminals.

[0015] Therefore, with such a media conversion function,
It enables calls between able-bodied people (voice input/output) and hearing-impaired people (text input/output), enables communication between different media, converts answering machine playback to text data, and enables listability and visibility. By doing so, it is possible to provide a multimedia communication system that can shorten playback time, make content confirmation easier, and further improve services.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is an embodiment of the present invention, and shows a schematic system configuration diagram. In the figure, 1 is a PBX (
2a and 2n are telephones which are voice terminals, and 3a and 3m are text data terminals such as personal computers. Furthermore, 4a and 4m are facsimile machines, and 5a and 5m are drawing communication terminals such as tablets. In addition, 6 is a signal processing device, and
It receives image signals such as text data and handwritten documents and stores them, converts audio signals to text data, converts text data to audio signals and outputs them, and separates image areas from image signals. to recognize character data,
It has functions such as voice synthesis, character recognition signal conversion, etc., such as converting this into text data, and signal and data recording and reproducing functions.

FIG. 2 is a functional block diagram of this signal processing device 6. In the figure, 60a is an audio terminal connection section for connecting an audio terminal, to which an audio terminal such as a telephone is connected. 60b is a text terminal connection section for connecting a text terminal, to which the text terminal is connected. 60c is a facsimile connection section for connecting a facsimile machine, and 60d is a drawing communication connection section for connecting a drawing communication terminal. 61 is the audio terminal connection section 6
0a is an A/D conversion circuit that converts the audio signal inputted from the A/D conversion circuit 61 into a digital signal, and 62 is an audio encoding unit that converts the digital signal converted by the A/D conversion circuit 61 into, for example, a PCM code. be.

Reference numeral 63 denotes an audio data storage section, which can store and read out the digital signal encoded by the audio encoder as audio data. Reference numeral 64 denotes a speech recognition analysis section, which recognizes the speech signal, converts the speech signal into corresponding text data, and outputs the converted text data. 65 is a text data storage unit that stores and reads out text data, and 66 is an audio decoding unit that decodes the audio encoded data read out from the audio data storage unit 63.

Further, 67 is the text data storage section 6.
A voice synthesis section 68 converts the text data read out from 5 into a digital voice signal, and a voice synthesis section 68 converts the digital voice signal output from the voice synthesis section 67 or the voice decoding section 66 into an analog voice signal. This is a D/A conversion circuit that outputs to the audio terminal connection section 60a.

A receiving section 68 receives text data sent from the text terminal connection section 60b, sends it to the text data storage section 65, and stores it therein. 69 is a transmitting section, and the text data storage section 65
The text data read out is sent to the text terminal connection section 60b.

A receiving section 70 receives the image signal sent from the facsimile connection section 60c, sends it to the facsimile data storage section 71, stores it there, and sends it to the image area separation recognition section 72. The image area separation recognition unit 72 separates the image area from the facsimile data, extracts a character data part, recognizes characters from this extracted part, converts it into text data, sends it to the text data storage unit 65, and stores it there. let Reference numeral 73 denotes an image encoding section, which converts the text data read from the text data storage section 65 into an image code and outputs the image code.

[0023] 74 is a transmitter, and the image encoder 7
3 or the facsimile data (image signal) read out from the facsimile data storage section 71 to the facsimile terminal connection section 60c.

Reference numeral 75 denotes an A/D conversion circuit that converts the analog handwritten character signal inputted from the drawing communication connection section 60d into a digital signal, and 77 converts the digital signal converted by the A/D conversion circuit 75 into a digital signal. This is a drawing data storage unit that stores and reproduces data. 76 is a handwritten character recognition unit, which recognizes characters based on the digital signal converted by the A/D conversion circuit 75, generates text data corresponding to the recognized characters, sends it to the text data storage unit 65, and stores it here. It is something to remember.

Further, 78 is an image encoding section, which encodes the text data read out from the text data storage section 65 into an image signal suitable for a drawing communication terminal, and 79 is an image encoding section. A D/D converting the read data from the drawing data storage section 78 or the drawing data storage section 77 into an analog drawing signal and outputting it to the drawing communication connection section 60d.
This is an A conversion circuit.

The signal processing device 6 having such a configuration is a PBX.
1 as shown in Figure 3. The PBX 1 accommodates extension terminals and office lines, and performs exchange connections between extension lines and between extension lines and outside lines. Line circuit 13-
1 to 13-n, a control section 14, a storage section 15, and the like. 16 is a maintenance terminal. 2 is a standard terminal (voice terminal) such as a telephone or electronic key telephone, 4 is a facsimile, 3 is a data terminal with a modem, 5m is a drawing communication terminal, and 16-1 to 16-m are office lines.

The communication path switch 11 performs exchange connections between extension lines, or from an extension line to an outside line (office line), and from an outside line to an extension line.
It is in charge of various overall controls, including the control of the exchange operation. The storage section 15 stores various control programs and tables of the control section 14, and data such as set function contents, and also functions as a database.

The maintenance terminal 17 is used to change the contents of the various tables and functions stored in the storage section 15 or the database, and to change, add, and delete data, and to change programs. It also has a keyboard and character display device.

Line circuits 13-1 to 13-m are connection interfaces provided for each extension, and serve as connection interfaces between the call path switch 11 and each extension terminal. The office line trunk circuits 12-1 to 12-m correspond to the communication path switch 11 and the corresponding office lines 16-1 to 16-m, respectively.
This circuit also performs incoming call detection, DC loop closure for line capture, etc. In addition, although not shown, a tone sending circuit is connected to the communication path switch 11 so as to be able to send out various signal sounds such as a telephone set.

These components in the exchange main body of the PBX 1 are connected to the control unit 14 via control lines and data bus lines, and are controlled by line circuits 13-1 to 13- in accordance with a control program stored in advance in the storage unit 15. Control is performed so that a communication path is formed between M and the office line trunk circuits 12-1 to 12-m, thereby enabling normal switching operations. This communication path uses time slots obtained by time-division control of communication path highways provided for uplink and downlink respectively, to each line circuit 12-1 to 12-m and each office line trunk circuit 13-1 to 13-m, It is assigned to the interface 12-n, its own transmission signal is inserted into the time slot of the connected partner, and the signal from the other party is extracted from the time slot assigned to the self, thereby communicating with the other party. It is the communication path switch 11 that provides such a communication path, and the timing control of the time slot is performed by the control unit 14.

Further, each line circuit 12-1 to 12-m, each office line trunk circuit 13-1 to 13-m, and interface 12-n are controlled by the control section 14 to control the time slots as required. It has functions such as inserting signals into time slots and extracting signals from self-assigned time slots.

The signal processing device 6 also has an interface.
It is connected to the communication path switch 11 and the control section 14 via the communication path switch 13-n, and operates under the control of the control section 14. The signal processing device 6 includes an audio terminal connection section 60a that sends and receives audio signals;
a text terminal connection section 60b for connecting a text terminal;
Facsimile connection section 60c for connecting a facsimile terminal
, and a drawing terminal connection section 60d for connecting a drawing terminal, the interface 13-n has the function of selecting these connection sections and distributing signals to be sent and received in accordance with the attributes of the other terminal to be connected. It is held by

In addition, in order to control signal conversion according to the attributes of the extension, the storage unit 15 stores a control program therefor and data such as attributes indicating the type of each extension as a database. Based on this, in the case of an incoming call to an extension, the signal processing device 6 converts the call to match the attribute according to the attribute of the other party, and also performs various controls such as storing the signal, reading it out, and outputting it. It's like this.

That is, in this system, in incoming calls from an outside line to an extension line, outgoing calls from an extension line to an outside line, and communications between extension lines, the signal processing device 6 uses the signal processing device 6 to generate the optimal signal according to the attributes of the calling terminal and the other party's terminal. It is made so that it can be processed and given. This control is based on a command from the terminal using a special number, etc., and the control unit 14 searches the database for attribute information of the calling/receiving terminal, and takes into account the attribute information from the calling terminal to determine the optimal signal format for each communication partner. When a media conversion request is made by controlling the signal processing device 6 as shown in FIG. The signal processing device 6 performs media conversion and outputs the optimal signal format for each of the calling terminal and the called terminal, and the exchange main unit outputs the converted signal for the calling terminal. The converted output for the called terminal is given to the corresponding calling terminal, and the converted output for the called terminal is given to the corresponding called terminal to control communication.

Furthermore, by registering the settings of the mailbox and answering function, the control unit 14 of the main body of the exchange transfers incoming calls to the registered extensions and mailboxes to the signal processing device 6 for storage and storage, and for receiving calls from terminals. In response to a read command, it is possible to command the reading of a mailbox or recorded message on an answering machine, or to negotiate each other's attributes with the other terminal at the start of communication between terminals. If they are different, call the signal processing device 6 and request media conversion, and then
It has a function of performing recognition operation training between terminals and controlling communication between the communication terminals after confirming the media conversion function of the signal processing device 6.

In this system having such a configuration, the exchange connection operation for normal outgoing and incoming calls is as follows. When there is an incoming call from an outside line, the control unit 14 detects the incoming call from the central office line trunk circuits 12-1 to 12-m, and if there is an extension terminal set as a representative, the call is received on the representative extension, and the call is sent in an extension group system. If so, the call is controlled to arrive at the extension of that target group, and if it is a PBX dial-in or extension dial-in, the call is controlled to arrive at the extension corresponding to the dialed number.

[0037] Then, the response from the extension terminal secures a communication path and enables communication between the two parties. When a call is made from an extension to an outside line, the control unit 14 searches for a vacant central office line trunk circuit 12-1 to 12-m and uses this vacant central trunk circuit to make a call. Securing a route and dialing the central office exchange. Then, when connected to the other party via the central office exchange, communication becomes possible. The extensions receive the extension number from the calling extension, control the incoming call to the line circuit of the corresponding extension number, and the connected terminal of that line circuit responds to secure the communication path and enable communication.

Although this is a case of receiving a call to a telephone which is a voice terminal, this system allows communication between terminals using different media.

That is, voice input from a telephone, handwritten character input from a facsimile (FAX) or a drawing communication terminal are converted into text data by the signal processing device 6,
The data may be stored or converted again into a format suitable for the attributes (voice terminal, text terminal, image communication terminal) of the communication terminal such as a telephone that is the communication partner and output. Data from text terminals is stored and transmitted in its original form.

Next, the functional operation of the signal processing device 6 will be explained based on FIG. Outputs from various media terminals are input to the signal processing device 6 via the exchange main body.
This input is performed via an interface 13-n in the main body of the exchange through a dedicated connection section for each medium (for each attribute).

In other words, handwritten character information is input via the facsimile connection unit 60c, and image area separation is performed by the image area separation recognition unit 72 from information such as telewriting codes such as MH and MR sent from the facsimile machine. Thereafter, only the character information is extracted, converted into text data, and outputted, which is sent to the text data storage section 65 and stored therein. Further, the received information such as the telewriting code is also sent to the facsimile data storage section 71, where it is stored in its original data form.

As an output, the text data storage section 65
The image encoding unit 7 reads out the text data stored in the image encoding unit 7.
3, where the text data is converted into a telewriting signal code format such as MH, MR, etc., and sent to the transmitter 7.
4 and output to the facsimile terminal via the facsimile connection section 60c and the main body of the exchange.

Further, the image signal sent from the drawing communication terminal is inputted via the drawing communication connection section 60d, and
The digital signal is converted into a digital signal by the /D conversion circuit 75 and then sent to the handwritten character recognition section 76, where the character pattern is recognized, converted into text data corresponding to the recognized characters, and sent to the text data storage section 65. be remembered. Also,
The signals from the drawing communication terminal that are converted into digital signals by the A/D conversion circuit 75 are simultaneously sent directly to the drawing data storage section 77 and stored therein. When the feedback to the drawing communication terminal is to return the one stored in the drawing data storage section 77, the read signal is sent as it is to the D/A conversion circuit 79, converted into an analog signal, and then connected to the drawing communication connection section 60d and the exchange. Output to the drawing communication terminal via the main body. Further, when outputting the data stored in the text data storage section 65 to a drawing communication terminal, the data read from the text data storage section 65 is sent to an image encoding 78 for image encoding, and then sent to a D/A conversion circuit 79. send out,
After converting it into an analog image signal, connect it to the drawing communication connection section 60d.
and output to the drawing communication terminal via the main body of the exchange.

Text data from the text terminal is input via the text terminal connection section 60b, and what is received by the receiving section 80 is given to the text data storage section 65 and stored there, and the read text data is sent to the transmitting section. 69 to the text terminal.

[0045] Regarding audio information, the audio terminal connection section 60a
A signal is inputted via the A/D converter circuit 61, A/D converted by the A/D converter circuit 61, and then sent to the audio encoder 62.
After being converted into a CM code, it is sent to the audio data storage section 63 and stored there, while the output of the A/D conversion circuit 61 is also sent to the speech recognition/analysis section 64, where it is converted into text data. After that, it is sent to the text data storage section 65 and stored there.

As an output, the text data stored in the text data storage section 65 is sent to the speech synthesis section 67, where the text data is synthesized into an audio signal corresponding to the text data, and then sent to the D/A conversion circuit 68. The data is sent, D/A converted, and then output to the telephone via the text terminal connection section 60b and the main body of the exchange.

Although not shown, a control circuit is provided in order to control each element of the signal processing device 6 and perform various controls in cooperation with the exchange.
Various controls are realized under the control of this control circuit.

As described above, the key point of the present invention is that speech and handwritten characters are recognized, analyzed, converted into text data, and stored, and text data is stored as is as text data. The text data obtained is read out, synthesized into a voice signal by a voice synthesis section 67 and output to a voice terminal, and converted into a telewriting signal or an image signal by an image encoding section 73 to a facsimile terminal. It is output as text data to a text terminal, and is converted to a telewriting signal or an image signal by the image encoding unit 78 and output to a drawing terminal. The feature is that feedback can be provided in the form of a signal. This allows communication even if the attributes of the communicating terminals are different.

The operation of the system of the present invention will be explained in more detail. FIGS. 4 to 7 are operation transition diagrams showing the operation sequence of this system and the method of checking the conversion result.

Among these, FIG. 4 shows the case of email communication and storage type communication such as message notification when the other party is absent or broadcast communication to multiple targets, and FIG. 5 shows the case of image input. In this case, FIG. 6 shows an example of voice input in real-time communication, and FIG. 7 shows an example of handwritten character input in real-time communication.

The operating procedure is the same for message notification when the other party is absent, email communication such as broadcast communication to multiple targets, storage type communication (Fig. 4), and image input (Fig. 5). To confirm the conversion result into data, the signal processing device 6 notifies each communication terminal of the converted data into the same form as the input. Furthermore, if the recognition result is unsatisfactory for the operator, the signal processing device 6
Alternatively, in order to improve recognition accuracy, it is also possible to designate an input pattern using voice guidance from the signal processing device 6 after input becomes possible, and then allow the original input after learning. .

That is, in the case of FIG. 4, when a special number indicating a mail request is dialed from the terminal, the control section 14 of the network (in this case, the main body of the exchange) calls the signal processing device 6, and thereby the built-in signal processing device 6 Under the control of a control circuit (not shown), the main body of the exchange automatically responds. In response to this, the exchange main body notifies the terminal of the response, and the control section 14 of the exchange main body also informs the terminal of the response.
5 to find out the terminal class of the calling terminal, and inform the calling terminal and the signal processing device 6 of the information on the checked terminal class. Then, the signal processing device 6 plays voice guidance to the operator of the calling terminal to inform the operator of the work procedure.
The operator of the calling terminal listens to this and follows the guidance to enter voice input. Voice input is performed by the operator of the calling terminal speaking the message he or she wants to convey to the other party.

This voice message is given as a voice signal to the signal processing device 6 via the exchange main body, and the signal processing device 6 converts the given voice signal into an A/D conversion circuit 61.
The digital signal is converted into a digital signal and then sent to the voice encoding section 62 and the voice recognition analysis section 64.The voice encoding section 62 encodes the voice and stores it in the voice data storage section 63, and the voice recognition and analysis section 64 performs voice recognition. The converted text data is given to the text data storage section 65 and stored as text data.

[0054] Once the voice input is completed, the operator of the calling terminal operates the terminal to issue a "result confirmation request." This "result confirmation request" is given to the signal processing device 6 via the exchange main body, and the signal processing device 6 reads out the text data previously stored in the text data storage section 65 in accordance with this request and provides it to the speech synthesis section 67.

The speech synthesis section 67 performs speech synthesis in accordance with the read text data, converts it into an analog signal in the D/A conversion circuit 67, and sends it to the main body of the exchange. Since the main body of the exchange sends this to the calling terminal, the calling terminal can listen to and confirm the voice reproduction of its own message stored in the signal processing device 6. Once the confirmation is completed, the operator of the calling terminal performs an operation to end the call. As a result, a disconnection instruction is sent from the terminal to the exchange main body, and upon receiving this, the exchange main body disconnects the terminal and the signal processing device 6.

As described above, a message can be stored in the signal processing device 6 from the voice terminal. If it is a broadcast notification, the main body of the exchange makes a call to each extension, and when the called terminal responds, reads the previously stored text data,
In addition, the attributes of the called terminal that responded are checked from the database and matched to those attributes, and this text data is converted into a signal that matches each attribute (i.e., if it is a voice terminal, it will be converted into a voice signal, if it is a text terminal, it will be converted into a text data signal). If it is a facsimile terminal or a drawing communication terminal, it will be sent as a signal converted to the code format of a telewriting signal). As a result, the called terminal receives the message in a signal suitable for itself, so it can reproduce and output the message, and inform the called terminal user of the message.

[0057] In the case of a mailbox or message, the exchange main body notifies the target terminal that the mailbox or message has arrived, and when the user of that terminal performs an operation to read the mailbox or message, the exchange main body Signal processing device 6
The system reads the stored text data of the received mail or message, checks the attributes of the terminal from the database, matches the attributes, and sends out the text data as a signal that matches the attributes. As a result, since the terminal side receives the message with a signal suitable for itself, it can reproduce and output the message, and can notify the called terminal user of the message.

In the case of FIG. 5 (image input), when a special number indicating a mail request is dialed from a facsimile or a drawing communication terminal, the control unit 14 of the network (exchange main body) learns from this special number that it is a mail request, and performs signal processing. The device 6 is called, and the signal processing device 6 automatically responds to the main body of the exchange. In response to this, the exchange main body notifies the terminal of the response, and the exchange main body also sends a response to its control unit 1.
4 searches the database of the storage unit 15 to check the terminal class of the calling terminal, and notifies the calling terminal and the signal processing device 6 of the information on the checked terminal class. Then the signal processing device 6
notifies the operator of the calling terminal that input is possible;
We will also send you guidance. If the terminal is a facsimile and the connection is manual, a handset is used, so this is sent by voice, and if the terminal is a facsimile and the connection is automatic, or if the terminal is a drawing communication terminal, it is encoded as an image and notified that input is possible. After receiving this guidance, the operator of the calling terminal starts inputting images according to the guidance. For image input, either the operator of the calling terminal reads the message he or she wants to convey to the other party on a manuscript (in the case of a facsimile) or draws it (in the case of a drawing communication terminal).
Do it by doing things.

This input signal is given as a telewriting signal or an image signal to the signal processing device 6 via the exchange main body, and in the signal processing device 6, the given telewriting signal is received by the receiving section 70 and the image area is determined. The telewriting signal is sent to the recognition unit 72 and the facsimile data storage unit 71, and the facsimile data storage unit 71 stores the telewriting signal as it is. The telewriting signal input to the image area recognition unit 72 cuts out the character area, extracts only character information, recognizes the character pattern, converts it into corresponding text data, and sends it to the text data storage unit 65. , memorize it here.

The image signal is converted into a digital signal by the A/D conversion circuit 75 and then sent to the handwritten character recognition section 76, where the character pattern is recognized and converted into text data corresponding to the recognized characters. The data is sent to the data storage section 65 and stored therein. In addition, the A/D conversion circuit 75
At the same time, the signals from the drawing communication terminal converted into digital signals are directly sent to the drawing data storage section 77 and stored therein.

[0061] Once the image input is completed, the operator of the calling terminal operates the terminal to issue a "result confirmation request." This "result confirmation request" is given to the signal processing device 6 via the exchange main body, and the signal processing device 6 reads out the text data previously stored in the text data storage section 65 according to this request, and provides it to the image encoding section 73. , here converts the text data into the code format of the telewriting signal, and transmits it to the transmitter 7.
4 and output to the facsimile machine (if the terminal is a facsimile machine), the data is sent to the exchange main body.

Further, when the terminal is a drawing communication terminal, the signal processing device 6 outputs the data stored in the text data storage section 65 to the drawing communication terminal, converts the data read from the text data storage section 65 into an image code. The image is sent to a conversion circuit 78 for image encoding, and then sent to a D/A conversion circuit 79, which converts it into an analog image signal and sends it to the exchange main body for output to a drawing communication terminal.

[0063] In this way, in response to the reproduced signal sent to the exchange main body, the exchange main body sends it to the calling terminal, so that the calling terminal converts its own message stored in the signal processing device 6 into an image. You can see and confirm what has been reproduced in physical form. Once the confirmation is completed, the operator of the calling terminal performs an operation to end the call. As a result, a disconnection instruction is sent from the terminal to the exchange main body, and upon receiving this, the exchange main body disconnects the terminal and the signal processing device 6.

As described above, a message can be stored in the signal processing device 6 from a facsimile or a drawing communication terminal. Then, if it is a broadcast notification, the main exchange makes a call to each extension, and if the called terminal responds, it reads the previously stored text data and also retrieves the attributes of the answered called terminal from the database. Check the attributes and convert this text data into a signal that matches each attribute (i.e., if it is a voice terminal, it will be converted into an audio signal, if it is a text terminal, it will remain as text data, and if it is a facsimile terminal or a drawing communication terminal, it will be converted into a signal that matches its attributes.) send the signal converted to the coded format of the signal). As a result, the called terminal receives the message in a signal suitable for itself, so it can reproduce and output the message, and inform the called terminal user of the message.

In the case of a mailbox or message, the exchange main body notifies the target terminal that the mailbox or message has arrived, and when the user of that terminal performs an operation to read the mailbox or message, the exchange main body Signal processing device 6
The system reads the stored text data of the received mail or message, checks the attributes of the terminal from the database, matches the attributes, and sends out the text data as a signal that matches the attributes. As a result, since the terminal side receives the message with a signal suitable for itself, it can reproduce and output the message, and can notify the called terminal user of the message.

[0066] In addition, if the handwritten characters are such that the character pattern cannot be recognized, or if the received original is in its original state,
When you want to playback, etc., dial a special number from your terminal, and the main body of the exchange will instruct the signal processing device 6 to make the request, and will retrieve the image data of the received original from the facsimile data storage section 71 and the drawing data storage section 77. Since it is read and output, by receiving this on a terminal and outputting it, it is possible to reproduce the original manuscript as it is. The same goes for voice; the received voice is encoded and stored in the voice data storage unit 65, so if it is difficult to hear or if you want to hear the caller's real voice, you can reproduce the original voice. You can listen to it.

[0067] The above was an example of the case where data is once stored in the signal processing device 6 and then distributed. Next, real-time calls,
An example of communication will be explained.

FIGS. 6 and 7 show operation sequences for real-time calls and communications, with FIG. 6 showing the case of voice input and FIG. 7 showing the case of handwritten character input. The operating procedure for both is the same, and if the attributes differ from each other during negotiation with the other terminal (negotiation: a function for negotiating and changing the terms of use of the terminal) at the start of communication, the signal processing device 6 is called. , request media conversion, and then
After the recognition operation is trained and the conversion function is confirmed, communication between the communication terminals is performed.

Among these, applied operations such as executing the input operation again when the recognition result is unsatisfactory and learning based on guidance from the signal processing device are the same as those shown in FIG. 4-1.

Furthermore, if the communication terminal has a text output function, the recognition result can be confirmed immediately, and the above confirmation operation becomes unnecessary.

That is, in the case of FIG. 6, the calling terminal (for example, a voice terminal) dials the telephone number of the other party. The network (in this case, the exchange itself) then calls the corresponding party. When the other party responds, the exchange notifies the calling terminal of the response, and then the exchange enters the call control phase and performs terminal class negotiation to determine whether attributes match, terminal usage conditions match, etc. Negotiations will be carried out. If the conditions do not match, each terminal on the calling and called sides issues a media conversion request to the exchange (automatically or manually).
. Upon receiving this, the controller 14 of the exchange main body calls the signal processing device 6 and controls it to issue a media conversion request. The signal processing device 6 that has received the call and the media conversion request automatically responds to the main body of the exchange. In response to this, the main body of the exchange notifies the terminal of the response, and also requests terminal class instructions from each of the calling and receiving terminals. Since the calling terminal is a voice terminal, it is notified that it is a voice terminal. Further, if the called terminal is a text terminal, the called terminal notifies that it is a text terminal. This notification can be performed by inputting a predetermined number for each terminal class using a dial key or the like. Of course, if the information on the connected terminals of each line is recorded in the database of the storage unit 15 of the exchange main body, the exchange main body will automatically search this database and check the terminal class of the calling terminal according to the negotiation result. It is also possible to inform the signal processing device 6 of the information on the checked terminal class.

[0072] Upon receiving the notification, the main body of the exchange notifies the signal processing device 6 of this terminal class information. Then, the signal processing device 6 enters a training phase, sends audio guidance to the calling terminal, and sends an input permission notification to the called terminal.

When entering the training phase, the operator of the calling terminal enters voice input for training. As a result, the signal processing device 6 converts the input audio signal into a digital signal using the A/D conversion circuit 61 and provides it to the speech recognition analysis section 64, where the speech recognition and analysis section 64 performs speech recognition and converts it into corresponding text data. is given to the text data storage section 65 and stored as text data. The text data stored in the text data storage section 65 is read out and given to the speech synthesis section 67.
7 performs speech synthesis corresponding to the read text data, converts it into an analog signal in a D/A conversion circuit 67, and sends it to the exchange main body. Since the exchange itself sends this to the calling terminal, the calling terminal can confirm that it has heard its own voice sent during the training phase.

On the other hand, when the called terminal enters the training phase, it inputs text. The input text data is sent to the signal processing device 6, received by the receiving section 80, and then stored in the text data storage section 65. Then, this is read and sent to the transmitter 74, and the main body of the exchange sends it to the called terminal, so that the called terminal can receive and confirm its own text sent in the training phase. It should be noted that if the signal processing device 6 has a function of synthesizing the read text in a speech synthesis section, recognizing the speech in a speech recognition analysis section, converting it into corresponding text data, and sending it to the transmission section 74, It becomes possible to train the situation when communicating with a voice terminal.

[0075] When the training is completed, the calling terminal and the called terminal each give a "start instruction". Then, the communication phase begins. When you enter this mode,
The main body of the exchange supplies the voice input from the calling side terminal to the signal processing device 6, and the signal processing device 6 converts the given voice signal into a digital signal by the A/D conversion circuit 61 and then gives it to the voice recognition analysis section 64. . The speech recognition analysis section 64 performs speech recognition and converts the resulting data into corresponding text data, which is then applied to the text data storage section 65 to be stored as text data. Then, this text data is immediately read out and sent to the transmitter 74, and the exchange main body sends it to the called terminal, so that the called terminal, which is a text terminal, converts the audio signal sent from the calling terminal, which is a voice terminal, into text. It can be received in the form of data.

Further, the text data from the called terminal is given to the signal processing device 6 by the exchange main body, and the signal processing device 6 receives this given text data in the receiving section 80 and stores it in the text data storage section 65. and store it as text data. Then read this immediately,
It is given to the speech synthesis section 67. The speech synthesis section 67 synthesizes a speech signal corresponding to the read text data, converts this into an analog signal in the D/A conversion circuit 67, and sends it to the main body of the exchange. Since the main body of the exchange sends this to the calling terminal, the calling terminal can listen to the text data output by the called terminal in the form of audio.

[0077] In this way, it becomes possible to communicate even if the calling side and the called side use different media.

[0078] When the communication is completed, the operator of the calling terminal performs an operation to end the call. As a result, a disconnection instruction is sent from the terminal to the exchange main body, and upon receiving this, the exchange main body disconnects the terminal and the signal processing device 6.

The above is the mutual communication between the voice terminal and the text terminal. Next, mutual communication between the drawing communication terminal and the text terminal will be explained. In this example, image input (however, only text input)
and text input interaction. In the case of FIG. 7, the calling terminal (for example, a video terminal) dials the telephone number of the other party. The network (in this case, the exchange itself) then calls the corresponding party. When the other party responds, the exchange main unit notifies the calling terminal of the response, and then the exchange main unit enters the call control phase and performs terminal class negotiation to determine whether the attributes match or not.
Negotiate whether the terminal usage conditions match. If the conditions do not match, each terminal on the calling and called sides issues a media conversion request to the switchboard (
automatic or manual). Upon receiving this, the controller 14 of the exchange main body calls the signal processing device 6 and controls it to issue a media conversion request. The signal processing device 6 that has received the call and the media conversion request automatically responds to the main body of the exchange. In response to this, the main body of the exchange notifies the terminal of the response, and also requests terminal class instructions from each of the calling and receiving terminals. Since the calling terminal is a voice terminal, it is notified that it is a voice terminal.

Further, if the called terminal is a text terminal, the called terminal notifies that it is a text terminal. This notification can be performed by inputting a predetermined number for each terminal class using a dial key or the like. Of course, if the information on the connected terminals of each line is recorded in the database of the storage unit 15 of the exchange main body, the exchange main body will automatically search this database and check the terminal class of the calling terminal according to the negotiation result. It is also possible to inform the signal processing device 6 of the information on the checked terminal class.

[0081] Upon receiving the notification, the main body of the exchange notifies the signal processing device 6 of the information of this terminal class. Then, the signal processing device 6 enters the training phase and sends an input permission notification to the calling terminal and the called terminal.

When entering the training phase, the operator of the calling terminal enters drawing input for training. As a result, the signal processing device 6 converts the input drawing signal into a digital signal using the A/D conversion circuit 75 and sends it to the handwritten character recognition section 76, where the handwritten character pattern is recognized and converted into text data of the recognized character. Convert. and,
This converted data is given to the text data storage section 65,
Store as text data. This stored data is immediately read out, image encoded by an image encoder 78, converted into an analog signal by a D/A converter circuit 79, and sent to the exchange main body. Since the exchange itself sends this to the calling terminal, the calling terminal can immediately check the quality of its own training drawing content sent during the training phase.

On the other hand, when the called terminal enters the training phase, it inputs text. The input text data is sent to the signal processing device 6, received by the receiving section 80, and then stored in the text data storage section 65. Then, this is read and sent to the transmitter 74, and the main body of the exchange sends it to the called terminal, so that the called terminal can receive and confirm its own text sent in the training phase.

Note that the signal processing device 6 has a function of synthesizing the read text into speech in the speech synthesis section, recognizing the speech in the speech recognition analysis section, converting it into corresponding text data, and sending it to the transmission section 74. If there is, it will be possible to train the situation when communicating with a voice terminal.

[0085] When the training is completed, the calling terminal and the called terminal each give a "start instruction". Then, the communication phase begins. When you enter this mode,
The main body of the exchange supplies the drawing input from the calling side terminal to the signal processing device 6, and the signal processing device 6 converts the given drawing input signal into a digital signal by the A/D conversion circuit 75 and then sends it to the handwritten character recognition section 76. give. Then, the handwritten characters are pattern recognized and converted into text data of the recognized characters. The converted data is then given to the text data storage section 65 and stored as text data. This stored data is immediately read out and sent to the transmitter 6.
9 to the exchange main body. The exchange itself sends this to the called terminal, and the called terminal outputs the received text data.

[0086] Furthermore, a transmission output in the form of text data is sent from the called terminal to the main body of the exchange. The main body of the exchange sends this to the signal processing device 6. In the signal processing device 6, this text data is received by the receiving section 80 and provided to the text data storage section 65, where it is stored as text data. This stored data is immediately read out and provided to the speech synthesis section 67.

The speech synthesis section 67 synthesizes a speech signal corresponding to the read text data, converts this into an analog signal in the D/A conversion circuit 67, and sends it to the main body of the exchange. Since the main body of the exchange sends this to the calling terminal, the calling terminal can listen to the text data output by the called terminal in the form of audio. In this way, it becomes possible to communicate even if the calling and called parties use different media.

[0088] When the communication is completed, the operator of the calling terminal performs an operation to end the call. As a result, a disconnection instruction is sent from the terminal to the exchange main body, and upon receiving this, the exchange main body disconnects the terminal and the signal processing device 6.

As explained above, this system connects various media, such as voice terminals for telephone calls, text terminals for data communication, and image communication terminals for exchanging image information, and enables communication between communication terminals using at least the same type of media. In a multimedia communication system that enables communication, there is a text data receiving means that receives and outputs text data, and a text data converter that receives an audio input signal, performs speech recognition from it, analyzes it, converts it into text data, and outputs it. an image recognition conversion means that receives an image information signal, extracts character information from it, recognizes a character pattern, converts it into text data corresponding to the recognition result, and outputs it; an image conversion output means that converts the output text data into image information and outputs it; and an output means that outputs the output text data as it is as a conversion output. When the signal processing device starts communication, the media of the terminal device to be communicated with is checked, and the output of the terminal device to be transmitted is given to the signal processing device, and the signal is converted into a signal compatible with the media of the terminal device to be received. It is configured to include an exchange means that controls the signal processing device and sends the converted and output signal to the receiving terminal device by giving the output of the transmitting terminal device to the signal processing device. At the start of communication, the exchange means checks the medium of the terminal device to be communicated with, and sends the output of the terminal device to be transmitted to the signal processing device, in order to convert it into a signal compatible with the medium of the terminal device to be received. In addition to controlling the signal processing device, the output of the transmitting terminal device is given to the signal processing device, and the signal processing device converts the output of the transmitting terminal device into text data, which is then transferred to the media of the receiving terminal device. The signal is converted into a corresponding signal format and outputted to the exchange means, and the exchange means sends the converted and output signal from the signal processing device to the receiving target terminal device.

[0090] In this way, this system first converts the output from the sending terminal into text data, converts the signal form of this text data to match the media of the receiving terminal, and sends it to the receiving terminal. Since mutual communication between various media such as voice, text, facsimile, and drawing communication is possible, the attributes of the communicating terminals do not matter, and there is no need to be aware of the type of the other terminal. can communicate with each other. Therefore, it becomes possible to make calls between able-bodied people (voice input/output) and hearing-impaired people (text input/output), as well as broadcast communications, message exchanges, and e-mail exchanges with any recipient, which is extremely convenient. In addition, the playback of the guard phone can be made into text data and can be viewed at a glance, which can shorten the playback time.
Furthermore, it is possible to configure a system that is extremely user-friendly for users, such as by making it easier to browse and reconfirm important messages.

[0091] Therefore, according to the present invention, it is possible to convert the medium to enable a conversation between a able-bodied person (voice input/output) and a hearing-impaired person (text input/output), or to enable communication between different media. It is extremely user-friendly, with features such as converting answering machines into text data, shortening the playback time by making it easier to check the content, and further improving the service. It can be a convenient multimedia communication system.

The present invention is not limited to the embodiments described above and shown in the drawings, but can be practiced with appropriate modifications within the scope of the invention. For example, in this embodiment, the signal processing device is installed in a private network centered on a PBX, but it is also possible to connect it to a public network as a public service. Furthermore, when connected to a digital network, the A/D, D/A, etc. shown in FIG. 3 are of course unnecessary.

[0093]

[Effects of the Invention] As described in detail above, according to the present invention, it is possible to make a call between a able-bodied person (voice input/output) and a hearing-impaired person (text input/output) by converting the medium. By enabling mutual communication, converting answering machine playback into text data and making it listable, the playback time will be shortened, and the content will be easier to confirm, thereby further improving the service. It is possible to provide a multimedia communication system that is extremely easy to use and convenient.

[Brief explanation of drawings]

FIG. 1 is a schematic block configuration diagram of an exchange system showing an embodiment of the present invention.

FIG. 2 is a block configuration diagram showing the configuration of a signal processing device according to the present invention.

FIG. 3 is a diagram for explaining the system according to the present invention, and is a block diagram showing a configuration centered on an exchange.

FIG. 4 is an operation transition diagram (in the case of voice input) for explaining registration operations for mail communication, messages, and broadcast notifications in the device of the present invention.

FIG. 5 is an operation transition diagram for explaining a registration operation in the case of image input in the apparatus of the present invention.

FIG. 6 is an operation transition diagram in real-time communication in the device of the present invention (in the case of mutual communication between a voice terminal and a text terminal).

FIG. 7 is an operation transition diagram in real-time communication in the device of the present invention (in the case of mutual communication between an image terminal and a text terminal).

FIG. 8 is a block diagram for explaining the configuration of a conventional system.

[Explanation of symbols] 1...Electronic exchange (PBX), 2...Telephone, 3...Data terminal, 4...Facsimile, 6...Signal processing device, 11...Call path switch, 13-1 to 13-m...Line circuit, 13 −
n...Interface, 14...Control unit, 15...Storage unit, 1
4...Maintenance terminal, 2...Extension telephone, 21m...
Extension terminal, 12-1, ~12-m...Trunk circuit, 16
-1, ~16-m... Office line, 60a... Audio terminal connection section, 6
0b...Text terminal connection section, 60c...Facsimile connection section, 60d...Drawing communication connection section, 61, 75...A/
D conversion circuit, 62...Speech encoding unit, 63...Speech data storage unit, 64...Speech recognition analysis unit, 65...Text data storage unit, 66...Speech decoding unit, 67...Speech synthesis unit, 68,
79...D/A conversion section, 68...Reception section, 69...Transmission section, 7
0... Receiving unit, 72... Image area separation recognition unit, 73... Image encoding unit, 74... Transmitting unit, 76... Handwritten character recognition unit, 77... Drawing data storage unit, 78... Image encoding unit.

Claims (4)

[Claims] [Claim 1] Multimedia that connects various media such as voice terminals for making phone calls, text terminals for data communication, and image communication terminals for exchanging image information to enable communication between communication terminals using at least the same type of media. In a communication system, a text data conversion means receives a voice input signal, performs voice recognition from it, analyzes it, converts it into text data, and outputs it, and a text data conversion means receives an image information signal and extracts character information from it, and recognizes a character pattern. an image recognition conversion means for converting the recognition result into text data corresponding to the recognition result and outputting the result; a speech synthesis means for converting the output text data into speech and outputting the same; A signal processing device comprising an image conversion output means that converts the image into a signal and outputs the image, and a signal processing device that checks the media of the terminal device to be the communication partner at the start of communication, gives the output of the terminal device to be transmitted to the signal processing device, and outputs the image to be received. The signal processing device is controlled to convert it into a signal compatible with the media of the terminal device, and the output of the transmitting terminal device is given to the signal processing device, so that the converted and output signal is sent to the receiving terminal device. a multimedia communication system comprising: an exchange means for transmitting data to a computer; [Claim 2] Multimedia that connects various media such as voice terminals for making phone calls, text terminals for data communications, and image communication terminals for exchanging image information to enable communication between communication terminals using at least the same type of media. In a communication system, a text data conversion means receives a voice input signal, performs voice recognition from it, analyzes it, converts it into text data, and outputs it, and a text data conversion means receives an image information signal and extracts character information from it, and recognizes a character pattern. an image recognition conversion means for converting the recognition result into text data corresponding to the recognition result and outputting the result; a speech synthesis means for converting the output text data into speech and outputting the same; a signal processing device comprising an image conversion output means for converting and outputting the converted text data; and an output means for outputting the outputted text data as it is as a converted output;
At the start of communication, the media of the terminal device to be communicated with is checked, the output of the terminal device to be transmitted is given to the signal processing device, and the signal processing device is controlled to convert it into a signal compatible with the media of the terminal device to be received. and an exchange means for transmitting the output of the transmitting terminal device to the signal processing device, thereby transmitting the converted and output signal to the receiving terminal device. [Claim 3] Multimedia that connects various media such as voice terminals for making phone calls, text terminals for data communications, and image communication terminals for exchanging image information to enable communication between communication terminals using at least the same type of media. In a communication system, a text data conversion means receives a voice input signal, performs voice recognition from it, analyzes it, converts it into text data, and outputs it, and a text data conversion means receives an image information signal and extracts character information from it, and recognizes a character pattern. an image recognition conversion means for converting the recognition result into text data corresponding to the recognition result and outputting the result; a storage means for storing the output text data; and a storage means for reading out the text data from the storage means, converting it into speech, and outputting the result. a signal processing device that serves as a communication partner at the start of communication; Checks the media of the terminal device, provides the output of the terminal device to be transmitted to the signal processing device, controls the signal processing device to convert it into a signal compatible with the media of the terminal device to be received, and transmits the output of the terminal device to be transmitted. 1. A multimedia communication system comprising: an exchange means for applying the output of the device to the signal processing device and transmitting the converted and output signal to a terminal device to be received. Claim 4: Multimedia that connects various media such as voice terminals for making phone calls, text terminals for data communications, and image communication terminals for exchanging image information to enable communication between communication terminals using at least the same type of media. In a communication system, a text data receiving means receives text data input, a text data converting means receives an audio input signal, performs speech recognition from it, analyzes it, converts it into text data and outputs it, and receives an image information signal. An image recognition conversion means extracts character information from this, recognizes a character pattern, converts it into text data corresponding to the recognition result, and outputs it, and the text data to be outputted and the text data received by the text data reception means are A storage means for storing, a speech synthesis means for reading the text data in the storage means, converting it into speech, and outputting it, and a speech synthesis means for reading the text data in the storage means and converting it into image information corresponding to the read text data. A signal processing device comprising an image conversion output means for outputting an image, an output means for reading text data from the storage means and outputting the read text data as it is as a converted output, and media of a terminal device that becomes a communication partner at the start of communication. The output of the transmitting terminal device is given to the signal processing device, and the signal processing device is controlled to convert it into a signal compatible with the media of the receiving terminal device, and the output of the transmitting terminal device is controlled. 1. A multimedia communication system comprising an exchange means for transmitting a signal converted and outputted by applying the signal to the signal processing device to a terminal device to be received.