JPH11177518A - Data communication device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To efficiently transmit a plurality of types of media data using a multiplexing process. In a data communication device, for example, an input unit inputs image data, audio data, and handwritten character image data from each unit of an input device, and a switching unit of a data conversion unit. Switches the type of media data to be transmitted from the input media data in accordance with the conditions set according to the type of media and the user's instruction input from the data operation unit 15. As a result, when two or more types of media are switched to transmission targets, the multiplexing unit of the data conversion unit 13 multiplexes the two or more types of media data, and the transmission unit 14 transmits the multiplexed data. When only one type of media is switched to the transmission target, the transmitting unit 14 transmits only the media data. The data output unit 12 reproduces and outputs the media data, and the data temporary storage area 16 temporarily stores, for example, untransmitted media data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a data communication apparatus for multiplexing and transmitting a plurality of types of media data,
In particular, a data communication device that realizes efficient transmission of a plurality of types of media data by switching the type of media data to be transmitted from input media data in accordance with a condition set according to the type of media. About. The present invention also relates to a data communication device for multiplexing and transmitting a plurality of types of media data,
In particular, the present invention relates to a data communication device that realizes efficient transmission of a plurality of types of media data by switching the type of media data to be transmitted from input media data in accordance with a user's instruction.

[0002]

2. Description of the Related Art For example, in a mobile communication device or a videophone device, a plurality of types of media data such as voices, images and characters are multiplexed and communicated. FIG.
1 shows a configuration example of a device that multiplexes and transmits image data and audio data as an example of a data communication device that multiplexes and transmits a plurality of types of media data. FIG. 2 also shows an example of the configuration of the receiving-side device that receives the data transmitted from the transmitting-side device via the communication path 95 together with the transmitting-side device.

In the transmitting apparatus shown in FIG. 1, an image encoding section 91a provided in an encoding section 91 compresses and encodes digitally converted input image data, and is provided in the encoding section 91. The audio encoding unit 91b compression-encodes the digitally converted input audio data, and the multiplexing unit 92 multiplexes the encoded image data and the encoded audio data to generate multiplexed data.

In the transmitting apparatus, the error correction encoder provided in the communication control unit 93 performs error correction encoding on the generated multiplexed data, and the modulator 94
Modulates the multiplexed data subjected to the error correction coding into a signal suitable for the communication channel 95, and transmits and outputs the modulated signal to the communication channel 95. The communication control unit 93 also performs data flow control, error detection processing, and the like.

In the receiving apparatus shown in FIG.
6 demodulates a signal received and input from the communication path 95 into multiplexed data, and an error correction decoder provided in the communication control unit 97 performs error correction decoding processing on the demodulated multiplexed data. Reference numeral 98 separates the multiplexed data subjected to error correction decoding into encoded image data and encoded audio data. In the above-described apparatus on the receiving side, the image decoding unit 99a provided in the decoding unit 99 decodes the separated coded image data, and the audio decoding unit 99b provided in the decoding unit 99. Decodes the separated encoded audio data, thereby reproducing and outputting the decoded audio data and image data, for example.

FIG. 8 shows an example of a temporal change in the transmission rate (bps) between image data and audio data in the multiplex communication described above. In FIG. 8, the horizontal axis indicates time and the vertical axis indicates time. The transmission rate is shown. FIG. 2 shows a case where the transmission rate 101 of image data and the transmission rate 102 of audio data are constant regardless of time.

As described above, when communication of a plurality of types of media data is performed, the plurality of types of media data are multiplexed and communicated, and data to be transmitted on the transmission side is transmitted by an information source code. On the other hand, encoded data received on the receiving side is decompressed and decoded in accordance with the compression method. By performing communication by compressing and encoding the media data in this manner, the amount of data that is actually transmitted from the transmitting side to the receiving side can be reduced. For example, as a method for compressing and encoding audio data, G.1 of ITU-T recommendation is used. 711,
G. FIG. 721, G.R. 723.1, G.C. 728, G.R. 729
Such a method is known.

[0008] For example, a JPEG method or the like is generally known as a method for compressing and encoding still image data, and an ISO standard MPEG1 or MPEG2 is used as a method for compressing and encoding moving image data. And ITU-T Recommendation H.264. 261, H .; 262, H .; 2
Methods such as 63 are known.

In the above-described compression encoding method for still image data and moving image data, image data is encoded according to substantially the same processing procedure. That is, specifically,
In these compression coding methods, a discrete cosine transform (DCT) is performed on image data, and the number of effective coefficients of DCT coefficients obtained by the transform is reduced using a quantizer to reduce DC
By performing entropy encoding on the T coefficient by an entropy encoder, compression-encoded image data is obtained. As the entropy encoder, for example, a Huffman encoder is used.

Further, in the method of decompressing and decoding image data encoded by the above-mentioned compression encoding method, the encoding image data is generally processed in the reverse order of the above-mentioned compression encoding processing. Decryption is performed. That is, specifically, after the encoded image data is entropy-decoded, the entropy-decoded data is inversely quantized, and the data obtained by the inverse quantization is subjected to an inverse discrete cosine transform, thereby performing decompression decoding. Obtained image data is performed.

[0011]

However, in the data communication apparatus as described above, although a plurality of types of media data can be multiplexed and transmitted, for example, the input status of each media data to be transmitted varies. Even if such a case, it was not possible to switch the transmission processing method according to such input conditions or the type of media, and thus it was not sufficiently realized to efficiently transmit a plurality of types of media data. There was a defect. Specifically, for example, in a situation where a fixed transmission rate is fixedly assigned to each of image data and audio data as shown in FIG. 8 above, the input situation of audio data and image data to be transmitted and Regardless of the communication status, etc., the transmission rate assigned to each media data is fixedly maintained during the communication, so that it is not possible to perform efficient communication corresponding to the above input status, etc. Did not.

In the above data communication device,
In some cases, media data to be transmitted is compression-encoded. In such a case, since the method of compression-encoding each type of media data is fixed to only one type, It is not possible to select and switch the means for compressing and encoding media data in accordance with the communication path conditions, the type of media, etc., and therefore, it is necessary to perform efficient data communication due to fluctuations in communication conditions, etc. There was a problem that the case where it was not possible occurred.

Further, for example, even if the data communication apparatus as described above is provided with a plurality of types of compression encoding means for media data, the user simply needs to pre-set the media data before starting the data communication. Only one type of compression encoding means can be selected, and the media data compression encoding means is dynamically selected according to the conditions of the communication path which fluctuates during data communication, user requests, and the like. I couldn't do that.

As described above, in the above-described data communication apparatus, the type of media to be transmitted is switched as the switching of the transmission processing method in accordance with, for example, the communication status, the type of media, or the user's instruction. In addition, there has been a demand that it is possible to switch means for encoding media data, thereby realizing efficient data communication. In addition, such a request is particularly strong in, for example, mobile communication using a communication path with low communication quality, and realizes comfortable data communication even when using such a communication path with low quality. The request to do so was particularly strong.

The present invention has been made to solve such a conventional problem, and when transmitting a plurality of types of media data by using a multiplexing process, a media to be transmitted from among the input media data. An object of the present invention is to provide a data communication device capable of efficiently performing communication of a plurality of types of media data by switching data types. More specifically, in the data communication device described above, the type of the media data to be transmitted is switched according to a condition set according to the type of the media or an instruction from the user.

Further, according to the present invention, when a plurality of types of media data are transmitted by using a multiplexing process, a means for compressing and encoding the media data is dynamically selected during communication, for example, so that a plurality of types of media data are transmitted. An object of the present invention is to provide a data communication device capable of efficiently performing data communication. More specifically, in the above-described data communication device, the above-described encoding unit is selected in accordance with a communication situation, a type of a medium, and a user's instruction.

[0017]

In order to achieve the above object, in a data communication apparatus according to the present invention, when transmitting a plurality of types of media data using a multiplexing process, the data is transmitted as follows. Transmission is performed by switching the type of media. That is, the switching unit switches the type of media data to be transmitted from the media data input from the input unit according to the condition set according to the type of the media, thereby switching two or more types of media data as transmission targets. In this case, the multiplexing unit multiplexes the two or more types of media data, and the transmitting unit transmits the multiplexed media data. Also,
When only one type of media data is switched as a transmission target by the switching unit, the transmission unit transmits the media data without multiplexing the media data.

Therefore, when three types of media data are transmitted, for example, any one type of media data is transmitted alone by setting switching conditions in advance according to the type of each media. In accordance with the setting conditions, a transmission processing method of multiplexing and transmitting any two types of media data and a transmission processing method of multiplexing and transmitting three types of media data. Thus, it is possible to efficiently transmit a plurality of types of media data.

Any condition may be used as the condition set in accordance with the type of the above-described medium. For example, a condition of a medium requiring immediacy while transmitting data of a medium not requiring immediacy may be used. When the data is input, a condition such as temporarily stopping the transmission of the data of the media not requiring the immediacy and transmitting the data of the media requiring the immediacy with priority can be used as the setting condition. .

Further, in the data communication device according to the present invention,
When transmitting a plurality of types of media data using the multiplexing process, transmission is performed by switching the type of media to be transmitted as follows. That is, the switching unit switches the type of media data to be transmitted from the media data input from the input unit in accordance with a user's instruction, and when two or more types of media data are switched as transmission targets, multiplexing is performed. The multiplexing unit multiplexes the two or more types of media data, and the transmitting unit transmits the multiplexed media data. When only one type of media data is switched as a transmission target by the switching unit, the transmitting unit transmits the media data without multiplexing the media data.

Therefore, when three types of media data are transmitted, for example, any one type of media data is transmitted alone in accordance with an instruction input by a user in a timely manner according to a communication condition that fluctuates during communication. Transmission is performed by switching between a transmission processing method, a transmission processing method of multiplexing and transmitting any two types of media data, and a transmission processing method of multiplexing and transmitting three types of media data. This makes it possible to efficiently transmit a plurality of types of media data.

Further, in the data communication apparatus according to the present invention,
When transmitting a plurality of types of media data by switching the type of media to be transmitted according to the conditions set according to the type of media as described above or the instruction of the user, the media data is compressed and encoded as follows. Select the means for conversion. That is, a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data are provided, and the encoding control means determines the communication status and the type of media from among the plurality of types of encoding means. The media data is encoded by an encoding means selected accordingly.

Therefore, for example, there are variations in communication conditions such as the quality of a communication path during data communication, and characteristics for each type of media, such as the need to quickly transmit voice data requiring immediacy. For example, means suitable for encoding media data can be dynamically selected and switched during communication, for example, from among a plurality of types of compression encoding means, thereby enabling transmission of a plurality of types of media data. It is possible to realize efficient operation.

The communication conditions and the types of media on which the encoding means for compressing and encoding the media data are selected are not limited to the specific examples described above. The selection of the encoding means may be performed based on the type.

In the data communication device according to the present invention,
When transmitting a plurality of types of media data by switching the type of media to be transmitted according to the conditions set according to the type of media as described above or the instruction of the user, the media data is compressed and encoded as follows. Select the means for conversion. That is, a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data are provided, and the encoding control means selects from the plurality of types of encoding means in accordance with a user's instruction. The encoding unit encodes the media data.

Therefore, in response to a user's instruction to transmit specific media data with a higher priority than other types of media data, for example, one of a plurality of types of compression / encoding means is used in accordance with such user's instruction. For example, means suitable for encoding the media data can be dynamically selected and switched during, for example, communication, thereby realizing efficient transmission of a plurality of types of media data. The mode of the user's instruction as a basis for selecting an encoding unit for compressing and encoding the media data is not limited to the specific example described above, and the selection of the encoding unit based on any user's instruction is not limited. May be performed.

Further, in the data communication device according to the present invention,
Audio data and other types of media data are transmitted as a plurality of types of media data by any of the data communication devices described above as follows. That is, further, the voice detecting means detects the input level of the voice input from the input means, and the switching means detects the input level based on the fact that the input level detected by the voice detecting means is equal to or less than a predetermined threshold. By switching the type of media data to the transmission target, the other type of media data is transmitted by the transmission unit using the silent section of the input voice.

Therefore, for example, while transmitting audio data requiring immediateness in a videophone or the like in preference to other types of media data, other data such as image data or the like is used by using the silent section of the audio. Can be transmitted, and thereby, it is possible to efficiently transmit a plurality of types of media data including the voice while ensuring the immediacy of the voice.

The predetermined threshold value set for the above-mentioned audio input level may be set to an arbitrary value according to the use condition of the apparatus. Can be set to a value at which it can be determined whether or not the user's voice has been input. With such a setting, the user's voice is transmitted while ensuring immediacy and the user's voice is interrupted. Other types of media data, such as image data, can be transmitted using silent sections.

Further, in the data communication device according to the present invention,
A plurality of types of media data including audio data and handwritten character image data are transmitted by any of the data communication devices described above as follows. That is, when voice data and handwritten character image data are input from the input means, the switching means switches both the input voice data and handwritten character image data to a transmission target, and the multiplexing means. Multiplexes both data, and the transmitting means transmits the multiplexed media data. In addition,
The handwritten character image data is, for example, data such as characters written on an input screen or the like by a user using an input pen or the like.

Therefore, for example, when a user writes a character or the like on an input screen and at the same time explains the character or the like by speaking voice, the handwritten character image data such as the character and the voice which is the user's voice are spoken. Since both data can be switched as a transmission target and the data of both media can be multiplexed and transmitted to the receiving side, the data of both media can be transmitted while guaranteeing the synchronization of voice and handwritten character image. It is possible to efficiently transmit a plurality of types of media data including the data.

[0032]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment according to the present invention will be described with reference to the drawings. FIG. 1 shows an example of a conceptual internal configuration of a data communication device according to the present invention. The data communication device 1 has a data input unit 11 for inputting media data, and a data input unit 11 for outputting media data. Output unit 12
And a data conversion unit 13 for performing a compression encoding process or a multiplexing process on media data, a communication control unit 14 for controlling a communication process, and a data operation unit 15 for receiving a user's instruction.
And a data temporary storage area 16 for storing media data.
And are provided. In this example, the data communication device 1 is provided with a processor, a memory, and the like that integrally control the processing in each of the processing units 11 to 16 described above, and the processor executes a control program to execute various processing. Is controlled.

The data input unit 11 has an input device 2 for taking in various types of media data, for example, and has a function of inputting media data obtained by the input device 2 as digital data into the data communication device 1. ing. In this example, the input device 2 includes a digital camera 2a that captures image data, a microphone 2b that captures voice data, and a handwriting input screen 2c that is integrated with a touch panel that captures handwritten character image data (handwritten data). Provided.

In this embodiment, the data input section 11 inputs media data from various input devices 2a, 2b, 2c,..., Thereby forming input means for inputting media data. . Note that, for example, the various input devices 2a, 2b, 2c,... Do not necessarily have to be integrated with the data communication device 1, and some or all of these input devices 2a, 2b, 2c,. May be provided outside the data communication device 1.

In this embodiment, the data input unit 1 described above is used.
1 includes an audio detection unit that detects an input level of an audio input from the input unit 11, an image detection unit that detects whether image data is input from the input unit 11,
A handwritten character image detecting means for detecting whether or not the handwritten character image data is input from the input unit 11 is provided, and has a function of notifying the data conversion unit 12 of such an input state.

The data output unit 12 has a function of outputting media data taken into the data communication device 1. In this example, the data output unit 12 outputs audio data from a speaker, and outputs image data or handwritten character image. The data is displayed and output using, for example, the above-described handwriting input screen 2c. In this example, a handwriting input screen 2 for capturing handwritten character image data
Although the image data and the handwritten character image data are configured to be displayed and output while also using c as a display screen, the screen for performing the display output may be provided separately from the handwriting input screen 2c.

The data conversion unit 13 has a function of performing, for example, a process of compressing and encoding the input media data, a process of converting the input media data into a communicable format, and the like. Has a function of performing a process of multiplexing the input media data and a process of storing the input media data in the data temporary storage area 16 in accordance with a condition set according to the type of the user or a user's instruction. I have. The communication control unit 14 has, for example, a function of performing error correction encoding processing on media data to be transmitted, a function of performing data flow control, and a function of modulating media data to be transmitted and transmitting and outputting it to a communication path. In addition, in the present example, it has a function of controlling the transmission processing of the media data in accordance with the conditions set according to the type of the medium and the instructions of the user. Note that the communication control unit 14 may be provided with, for example, a function of performing an error detection process.

Here, in this example, the type of media to be transmitted is switched according to the conditions set according to the type of media by the processing of the data conversion unit 13 and the communication control unit 14 described above, so that Switching means for switching the type of media data to be transmitted from the input media data in accordance with the set conditions. In this example, the data conversion unit 1 described above is used.
By switching the type of media to be transmitted according to the user's instruction by the processing of the communication control unit 3 or the communication control unit 14, switching means for switching the type of media data to be transmitted from the input media data according to the user's instruction is configured. .

In this embodiment, when two or more types of media data are to be transmitted simultaneously, the data conversion unit 13
Multiplexes the two or more types of media data to form a multiplexing unit that multiplexes two or more types of media data to be transmitted by the switching unit. Further, in this example, the communication control unit 14 transmits one type of media data or multiplexed media data, thereby forming a transmission unit for transmitting the media data.

The data operation unit 15 has a function of receiving an instruction from a user. In this example, the data operation unit 15 includes buttons provided on the above-described handwriting input screen 2c. The data operation unit 15 may have any configuration as long as it can receive an instruction or the like from a user. For example, the data operation unit 15 may be configured by a keyboard, a mouse, or the like. it can. The data temporary storage area 16 is composed of, for example, a memory that stores data, and has a function of storing media data.

With the above configuration, in the data communication device 1, when inputting and transmitting a plurality of types of media data, the data communication device 1 transmits the media data according to the conditions set according to the type of media or the user's instruction. The type of media data to be transmitted can be switched. For example, when one type of media data is switched as a transmission target, the media data is transmitted without being multiplexed, while two or more types of media data are transmitted. When the switching is performed, the two or more types of media data are multiplexed and transmitted. Thus, the data communication device 1 can efficiently transmit a plurality of types of media data while guaranteeing the immediacy required for each media, for example, according to the input status and communication status of each media data. Can be realized.

Although the function of transmitting a plurality of types of media data by the data communication device 1 has been described above, in the data communication device 1, for example, the communication control unit 14 demodulates the media data received from the communication path. The data conversion unit 13 performs an error correction decoding process, and the data conversion unit 13 performs a separation process of the received multiplexed data and a decompression decoding process of the encoded data. By outputting and outputting the output unit 12, a process of receiving and reproducing a plurality of types of media data can also be performed. In the present invention, the data communication device 1 may be provided with only a function of transmitting a plurality of types of media data, or may be provided with a function of receiving a plurality of types of media data together with the function of the transmitting side. It may be. Further, for example, a data communication device 1 is configured by a transmission device having a function on the transmission side and a reception device having a function on the reception side, and a plurality of data communication devices are connected between the transmission device and the reception device via a communication path or the like. It is also possible to adopt a configuration in which different types of media data are communicated.

Next, the configurations of the data conversion unit 13 and the communication control unit 14 of the above-described data communication device 1, which are essential parts of the present invention, will be described in more detail. FIG. 2 shows a configuration in which the encoding unit 21 and the multiplexing unit 22 included in the data conversion unit 13 and the communication control unit 14 transmit and output media data to the communication path 27. FIG. 2 shows the configuration of a communication control unit 32, a demultiplexing unit 33, and a decoding unit 34 for receiving media data from the communication path 27 and the like. In the configuration shown in the figure, the modulator 23 for performing the modulation process is provided separately from the communication control unit 14, and the demodulator 31 for performing the demodulation process is provided with the communication control unit 32.
And the case where it is provided separately. Further, any type of communication path 27 may be used,
For example, it may be a wired communication path or a wireless communication path.

In this example, a case is shown in which image data, audio data, and handwritten character image data are communicated using a multiplexing process as a plurality of types of media data. First, the configuration of a device that transmits media data to the communication path 27 will be described. The encoding unit 21 includes an image encoding unit 21a that compresses and encodes image data, an audio encoding unit 21b that compresses and encodes audio data, and a data encoding unit 21c that compresses and encodes handwritten character image data. have.

The image encoding unit 21a has a plurality of types of encoding means for compressing and encoding image data, and has a function of outputting encoded image data to the multiplexing unit 22. In this example, a plurality of types of encoding means for compressing and encoding image data in the image encoding unit 21a include a plurality of types of encoders P1, encoder P2, encoder P3,.
・ ・ It is composed of As these plural kinds of encoding means, plural kinds of means having different characteristics are provided. For example, means which exerts an effect such that the influence of an error is small when image data is encoded at a high compression rate. Various encoding means are provided, such as a means that exerts an effect such that the effect of an error is small when image data is encoded at a low compression rate.

The audio encoding section 21b has a plurality of types of encoding means for compressing and encoding the audio data, and has a function of outputting the encoded audio data to the multiplexing section 22. In this example, a plurality of types of encoding means for compressing and encoding audio data in the audio encoding unit 21b include a plurality of types of encoders S1, encoders S2, encoders S3,.
・ ・ It is composed of As these plural kinds of encoding means, there are provided plural kinds of means having different characteristics, for example, means exhibiting effects such as a small influence of an error when audio data is encoded at a high compression rate. Various encoding means are provided, such as a means that exerts an effect that the influence of an error is small when audio data is encoded at a low compression rate.

The data encoding section 21c has a plurality of encoding means for compressing and encoding handwritten character image data, and has a function of outputting the encoded handwritten character image data to the multiplexing section 22. It has. In this example, a plurality of types of encoding means for compressing and encoding handwritten character image data are composed of a plurality of types of encoders H1, H2, H3,
... As these plural kinds of encoding means, plural kinds of means having different characteristics are provided. For example, when the handwritten character image data is encoded at a high compression rate, the effect of an error is small. Various encoding means are provided, such as a means for exhibiting the effect and a means for exhibiting the effect that the influence of an error is small when the handwritten character image data is encoded at a low compression rate.

The encoding units 21a, 2a
In the encoding unit 21 having the image encoding units 21b and 21c, for example, according to an instruction signal input from the communication control unit 14 described later or a user instruction input from the data operation unit 15, a plurality of units provided in the image encoding unit 21a are provided. Types of encoders P1, P
., P3,..., The image data is encoded by an encoder selected from
1b, a plurality of types of encoders S1, S2, S
The encoding process of the audio data is performed by the encoder selected from 3,... And the data encoding unit 21c
, A plurality of types of encoders H1, H2, H3,.
It has a function to encode the handwritten character image data by the encoder selected from.

Here, with respect to the manner in which the encoding unit 21 selects a means for compressing and encoding each media data in accordance with a user's instruction, for example, the setting of the selection method is stored in the encoding unit 21 in advance. ing. As a specific setting example, for example, when the user instructs to increase the communication speed of specific media data, while selecting an encoding unit that further increases the compression ratio of the specific media data,
When the user instructs to reduce the communication speed of the specific media data, the user selects a coding unit that suppresses the compression rate of the media data. If an instruction is given to secure a band wider than the type of media data and give priority to the communication of the specific media data, an encoding unit that increases the compression rate of the other type of media data is selected and the other type is selected. Settings such as maintaining the substantial communication speed of the media data of the type are performed.

It should be noted that the selection of an encoder for each media data does not necessarily require that only one type of encoder be selected from among a plurality of types of encoders. A configuration may be used in which two or more types of encoders are selected accordingly, and media data may be encoded by combining the selected two or more types of encoders. Also, the method of selecting the encoding means of each media data in accordance with the user's instruction is not necessarily limited to the method described above, and is not limited to the specific examples described above. Various selection methods may be set based on the usage status and the like.

In this example, as described above, the encoding unit 21 performs the encoding process of the media data by the encoding unit selected in accordance with the user's instruction, so that the user can select one of the plural types of encoding units. An encoding control unit that encodes the media data by the encoding unit selected according to the instruction is configured.

The encoding section 21 also encodes encoded image data (encoded image data), encoded audio data (encoded audio data), and encoded handwritten character image data (encoded When outputting the writing character image data) to the multiplexing unit 22, encoding control data including data such as the type of encoding means (encoder) and encoding parameters for encoding each of these media data is output. The multiplexing unit 2
2 is provided. Note that the encoded control data is transmitted and output to the communication path 27 via the multiplexing unit 22, the communication control unit 14, and the modulator 23.

The multiplexing unit 22 includes a multiplexing control unit A24 for controlling the multiplexing process and a multiplexer 25 for performing the multiplexing process.
And a multiplexing control unit B26 that outputs media data to the communication control unit 14. The multiplexing control unit A24 includes:
The multiplexing unit 22 has a function of controlling a multiplexing process in the multiplexing unit 22. In this example, a medium to be transmitted from the input media data according to a condition set according to the type of the medium or an instruction from the user. A process for switching the type of data is performed. For example, when only one type of media data is switched to the transmission target, the multiplexing control unit A24 outputs the one type of media data (single data) to the multiplexing control unit B22, while outputting two or more types of media data. When the data is switched to the transmission target, the two or more types of media data are output to the multiplexer 25.

In the multiplexing control unit A24, for example, the input status of each media data detected by the voice detection unit or the handwritten character image detection unit provided in the data input unit 11 and the communication control unit 14 described later The type of media data to be transmitted is switched based on the detected communication status and the like.

As a specific example of the condition set according to the type of the medium, in this example, when a sound having a level higher than a predetermined threshold is input during the transmission of the image data, the transmission of the image data is temporarily stopped. Prioritize transmission processing of voice data, or if a handwritten character image is input during transmission of image data, temporarily suspend transmission of image data and prioritize transmission processing of handwritten character image data If, for example, a voice and a handwritten character image at a level higher than a predetermined threshold value are input, the data of both media may be multiplexed and transmitted, or when the transmission process of the image data ends. Is set to switch to transmission processing of other types of media data such as audio.

The multiplexing control unit A24 switches the type of media data to be transmitted based on, for example, a user instruction input from the data operation unit 15. The conditions set according to the type of media and the conditions for switching the type of media data to be transmitted based on a user's instruction are stored in advance in, for example, a memory provided in the multiplexing control unit A24. .

The multiplexer 25 has a function of multiplexing two or more types of media data input from the multiplexing control unit A24, and multiplexes the multiplexed data obtained by the multiplexing. It has a function of outputting to the control unit B26. The multiplexing control unit B26 has a function of outputting single data input from the multiplexing control unit A24 and multiplexed data input from the multiplexer 25 to the communication control unit 14.

The communication control unit 14 has a function of performing error correction coding processing on single data or multiplexed data input from the multiplexing control unit B 26 and outputting the processed data to the modulator 23. Further, in this example, the communication control unit 14 is provided with means for detecting a communication state such as the quality of the communication path 27 and a communication state of single data and multiplexed data. A function is provided for notifying the multiplexing control unit A24 of the situation.

Further, in this example, the communication control unit 14 performs an encoding process performed by the encoding unit 21 according to the detected communication status and the type of each medium such as an image, a voice, and a handwritten character image. It has a function to control. In particular,
The communication control unit 14 selects an encoder suitable for the communication status and the characteristics of the image data from among the plurality of types of encoders P1, P2, P3,... Provided in the image encoding unit 21a. A selected instruction signal is output to the encoding unit 21 instructing that the selected encoder encodes the image data.

In the communication control section 14, a plurality of types of encoders S1, S2, S2 provided in the speech encoding section 21b are provided.
3. An appropriate encoder is selected from among 3,... According to communication conditions, characteristics of audio data, and the like, and an instruction signal designating encoding of audio data by the selected encoder is encoded. 21. In the communication control unit 14, a plurality of types of encoders H provided in the data encoding unit 21c are provided.
An encoder suitable for communication conditions and the characteristics of handwritten character image data is selected from among 1, H2, H3,..., And the handwritten character image data is encoded by the selected encoder. An instruction signal designating this is output to the encoding unit 21.

In the above-mentioned case, it is not always necessary to select only one type of encoder from among a plurality of types of encoders in selecting an encoder for each media data. For example, a configuration is used in which two or more types of encoders are selected according to communication conditions and the like and media data is encoded by combining the selected two or more types of encoders. You may.

As the above-mentioned instruction signal, the instruction signal for image data, the instruction signal for voice data, and the instruction signal for handwritten character image data are not necessarily separate signals from the communication control unit 14 and the encoding unit. It is not necessary to use a configuration that is output to the communication control unit 21 to the encoding unit 21, for example, instructions for two or three types of media data are collectively included in one instruction signal. Alternatively, a configuration of the instruction signal such as the instruction signal may be used.

As for the manner in which the communication control unit 14 selects a means for compressing and encoding each media data according to the communication status and the type of media, for example, the setting of the selection method is stored in advance in the communication control unit 14. Have been. As a specific setting example of the communication status, for example, when the communication speed of the media data is reduced, an encoding unit that increases the compression rate of the media data is selected, while the communication speed of the media data is increased. Is set to select an encoding unit that suppresses the compression rate of the media data. Also, when the communication quality of the media data on the communication path 27 is good, the effect is exhibited when the media data has a low compression rate. On the other hand, when the error rate (error occurrence rate) of the media data in the communication path 27 is high, a setting is made such as selecting an encoding means exhibiting an effect at a high compression rate of the media data. Is performed.

As for the type of media, for example, even if some errors occur in several frames of the moving image data, the effect on the visibility of the entire moving image is small, For still image data, it is necessary to ensure that the errors that occur are noticeable, and for example, for video conferencing equipment, the immediacy of audio data is guaranteed even if there are some errors compared to image data. For example, when a user talks while writing characters in a video conference, for example, there is a need to ensure the immediacy and simultaneousness of voice data and handwritten character image data. A method for selecting an encoding unit for each media data is set based on a difference in characteristics between media and the like.

The method of selecting the media data encoding means according to the communication status and the type of media is not necessarily limited to the method described above, and is not limited to the specific examples described above. Based on the processing time required to compress and encode the media data, the compression ratio that can be achieved, and various factors such as the error resistance of the encoded media data,
Various selection methods may be set depending on the usage status of the system. In addition, for example, before the communication of the media data is started, a transmission rate used for communication between the transmitting device and the receiving device is determined, and a media data encoding unit is selected based on the transmission rate. It is also possible to use such a selection method.

In this example, as described above, the communication control unit 14
Controls the encoding unit 21 to perform encoding processing of the media data by the encoding unit selected according to the communication state and the type of the medium, so that the communication state and the media Encoding control means for encoding the media data is constituted by the encoding means selected according to the type.

Here, the means for detecting the communication status such as the quality of the communication channel 27 may not necessarily be provided in the communication control unit 14. In other words, the encoding unit 21 and the multiplexing control unit A24 are required to detect the communication status. Any configuration can be used as long as the configuration can be grasped. There is no particular limitation on the method of detecting the communication status, and any detection method may be used. The modulator 23 has a function of modulating multiplexed data or single data input from the communication control unit 14 into a signal suitable for the communication path 27 and transmitting and outputting the signal to the communication path 23.

With the above-described configuration, the transmission-side device transmits a plurality of types of media data to the communication path 2 using multiplexing processing.
7, the communication status and the type of media from among a plurality of types of compression encoding means provided for the data of each media such as images, voices, and handwritten character images, according to the user's instruction. For example, means suitable for encoding each piece of media data can be dynamically selected and switched during communication, for example, whereby communication of a plurality of types of media data can be efficiently performed.

In transmitting the plurality of types of media data to the communication path 27 by using the multiplexing process, the transmitting side device described above, in accordance with the conditions set according to the type of media and the instructions of the user. The type of media data to be transmitted can be switched. Here, a specific example of the switching process performed by the above-described transmitting device based on the setting conditions of the present example will be described. In FIG. 3, when only image data is transmitted as single data, for example, the user newly or additionally inputs handwritten character image data from the handwriting input screen 2c using the input pen 41. The example of the procedure of the switching process in the case is shown.

FIG. 3A shows a digital still camera 2.
a, a plurality of blocks 42, 43, 44, including header information etc.
.. Are transmitted to the communication path 27 from the data communication device 1 (in this example, the transmitting device), and the image data transmitted as a single data includes a header and the like. The process of adding a block to form a block is performed by, for example, the multiplexing unit 22 and the communication control unit 14 described above.

Each block 42, 43, 44 of the image data
In the figure, a header file 45 including header information and the like is added to the head of the data 46 of the image itself, and a count flag 47 and an end flag 48 are added behind the data 46 of the image itself. Here, the header file 45 includes information such as a transmission destination address and synchronization timing necessary for performing communication, and the count flag 47 includes a total number of blocks into which the image data is divided and the total number of blocks. The information includes information such as the number of the block in the block, and the end flag 48 attached to the end of the block indicates the end position of the block.

When the handwritten character image data is input during the transmission of the image data, the transmitting device temporarily suspends the transmission of the image data and converts the input handwritten character image data into a single file. Performs interrupt processing for transmission as data. Here, for example, the timing at which the transmission switching from the image data to the handwritten character image data is performed by the interruption process according to the change of the input state is performed by the communication control unit 1.
4 is an end flag 48 included in each block of image data
Is detected, the position of the block break is used as the timing of the switching process. The block 44 of untransmitted image data is stored in the data temporary storage area 16 after being processed by, for example, the data conversion unit 13.

When such a switching process is performed, FIG.
As shown in (b), following the block 43 of the transmitted image data, the handwritten character image data is divided into a plurality of blocks 51, 52,... And transmitted to the communication path 27, for example. . Note that the header files and the like added to the blocks 51 and 52 of the handwritten character image data are the same as in the case of the above-described image data, and thus description thereof will be omitted.

Further, when the handwritten character image data is transmitted in this manner, for example, as shown in FIG. 3C, the tip of the pen 41 for input from the handwriting input screen 2c integrated with the touch panel. When the non-contact state in which the tip of the pen 41 is not in contact with the screen 2c continues for a certain period of time, the non-contact state is determined by the handwritten character image detecting means provided in the data input unit 11. As a trigger (trigger), the data conversion unit 13 determines that the input of the handwritten character image data is completed, and controls the communication control unit 14 to restart the transmission process of the image data. That is, FIG.
As shown in (d), following the blocks 53 and 54 of the handwritten character image data transmitted as described above, the communication control unit 14 stores the untransmitted image data stored in the data temporary storage area 16. Is transmitted to the communication path 27.

The predetermined period of time in the non-contact state between the input pen 41 and the handwriting input screen 2c is stored in advance in, for example, the data conversion unit 13 or the like. In addition, an arbitrary value may be set as the fixed time in the non-contact state. In short, it is sufficient to set a value that determines that the input of the handwritten character image data has been completed.

In the above-described transmitting apparatus, for example, when voice data having an input level larger than a predetermined threshold is input during transmission of image data, the above handwritten character image data is input. In the same manner as described above, the image data transmission process is temporarily stopped, the image data for the untransmitted image data is stored in the data temporary storage area 16, and the input audio data is switched to the transmission target, and the audio data is transmitted to, for example, a plurality of blocks. Divide and send.

At this time, as a trigger for switching from the transmission processing of the audio data to the transmission processing of the image data again, for example, a state in which the input level of the audio input from the microphone 2b is equal to or lower than the predetermined threshold continues for a certain period of time. Is used.
That is, when the state in which the input level of the voice detected by the voice detection unit is equal to or lower than the predetermined threshold has continued for a certain period of time, the data conversion unit 13 determines that the voice input has ended, and sets the communication control unit 14 to Control is performed to restart image data transmission. Note that the predetermined threshold value for detecting the silent section of the voice as described above and the above-mentioned fixed time are previously stored in, for example, the data conversion unit 13. In addition, any value may be set as each of the predetermined threshold value and the predetermined time period. In short, it is only necessary to detect whether or not a voice to be transmitted has been input.

In the transmitting apparatus, for example, when both voice data and handwritten character image data having an input level larger than a predetermined threshold are input during transmission of image data, the handwritten As in the case where only character image data and audio data are input, the transmission processing of the image data is temporarily stopped, the image data for the untransmitted data is stored in the data temporary storage area 16, and the input audio data and The writing character image data is switched to a transmission target, and both data are divided into a plurality of blocks, for example, and then multiplexed and transmitted.

At this time, as a trigger for switching from the transmission processing of the audio data and the handwritten character image data to the transmission processing of the image data again, the input of the audio data larger than the predetermined threshold is completed in the same manner as described above. It is used when it is determined that the input of the handwritten character image data has been completed while it is determined that the input has been completed.

In the above-described transmitting apparatus, for example, when the image data transmission processing is completed, the audio data is switched to the transmission target regardless of the input state of the audio data. Here, as the time point at which the transmission of the image data ends, for example, the communication control unit 14 determines whether the count flag 47 or the end flag 48
Is detected, and the time when the last block of the image data to be transmitted is transmitted to the communication path 27 is used. At this time, the transmission process from the image data to the audio data is switched.

In the above-described transmitting apparatus, for example, when handwritten character image data is input during transmission of voice data, when voice data is input during transmission of handwritten character image data, When the data of these two media are input at the same time, the data of both these media are switched to the transmission target, and both media data are multiplexed and transmitted to the communication path 27.

As described above, in the data communication device 1 of the present embodiment, the input voice data and handwritten character image data can be transmitted in real time while ensuring the immediacy, and, for example, input simultaneously. The voice data and the handwritten character image data can be transmitted while ensuring synchronism. Further, in the data communication device 1 of the present example, while voice data and handwritten character image data are not input, image data is transmitted in real time, while voice data and handwritten character image data are input. By temporarily storing the untransmitted image data in the memory, the transmission of the untransmitted image data accumulated after the transmission of the voice data and the handwritten character image data is completed can be resumed.

A specific example of the switching process performed by the above-described transmitting apparatus in accordance with a user's instruction will be described.
FIG. 4 shows the above-described transmitting device (data communication device 1).
The device 1 has a display screen 2c integrated with a touch panel, a transmission button 61 for inputting a transmission instruction, a reception button 62 for inputting a reception instruction, and various modes. Mode button 63 for switching
And a type button 64 for selecting the type of media data
And a status display area 65 for displaying a status such as a processing status. In this example, as described above, the handwriting input screen is also used as the display screen 2c.

For example, when transmitting image data captured by the digital still camera 2a, first, when the user presses the type button 64, a pop-up menu 66 is displayed. "And" handwriting (handwritten character image)
Is displayed, and “image” is selected as a transmission target, for example, by a user's instruction. Further, when the mode button 63 is pressed by the user, for example, encoding means such as “sequential method” or “progressive method” for image data is displayed,
Information on the selected encoding method is notified to the data conversion unit 13 according to a user's instruction, and the data conversion unit 13 selects and switches, for example, an optimal compression encoding unit.

After the image data to be transmitted displayed on the display screen 2c is confirmed by the user or, for example, when a plurality of image data is input, the image data to be transmitted is selected by the user. Then, when the transmission button 61 is pressed, transmission processing of image data to be transmitted is started. Send button 6
When 1 is pressed to start the transmission processing, a display such as "image data communication" is displayed in the status display area 65, for example.

The "sequential method" is a method in which image data is encoded by one scan in, for example, JPEG, and the "progressive method" is a method in which image data is encoded in a plurality of times in JPEG, for example. This is a method of encoding by scanning and performing transmission processing and the like sequentially from rough image information.

For example, when audio data, handwritten character image data, and multiplexed data of both data are transmitted and image data is selected as a transmission target as described above, a transmission button is transmitted as described above. When the button 61 is pressed, the transmission process of the transmitted audio data or the like is temporarily stopped, and the transmission process of the image data is started. Further, the transmission process and the reception process of each type of media data are controlled not only when the above-described image data is transmitted but also when various buttons 61 to 64 are pressed by the user.

As described above, in the data communication apparatus 1 of the present embodiment, for example, image data is transmitted to the receiving side by switching the media data to be transmitted in accordance with the conditions set according to the type of the medium and the instruction of the user. After that, it is possible to perform real-time conversation while viewing the same image on the transmission side and the reception side, and to perform communication of the handwritten character image in real time. The present invention can be applied to a telephone device or the like to realize an efficient videophone function.

FIG. 5 shows an example in which the data communication apparatus 1 of the present embodiment is applied to a portable telephone and the type of media data to be transmitted is switched as described above. 5 shows an example of a change over time in the transmission rate when the audio data and the handwritten character image data are transmitted after being compression-encoded and multiplexed together with the transmission data. In the figure, the horizontal axis indicates time and the vertical axis indicates transmission rate (bps). The data transmission rate of the mobile phone is, for example, 9600 bp.
s.

In the example shown in FIG. 7
8000 bp compressed and encoded by the 29 encoding method
s audio data 71 and compression-encoded handwritten character image data 72 are multiplexed and transmitted. In this case, the handwritten character image data 72 is assigned a transmission rate of 1600 bps. Next, for example, when the image data is switched to a transmission target, the compression-encoded image data 73 is transmitted as a single data at a transmission rate of 9600 bps. Then, for example, when the transmission processing of the image data 73 ends, the 8000 bps audio data 74 and the 1600 bps handwritten character image data 75 are multiplexed and transmitted. The transmission rate assigned to each piece of media data is assumed to be fixed.

FIG. 6 shows an example in which the data communication apparatus 1 of the present embodiment is applied to a portable telephone, and the type of media data to be transmitted is switched as described above. An example of a change over time in a transmission rate in a case where the audio data and the handwritten character image data are transmitted after being compression-encoded and transmitted in a progressive manner while being compressed and multiplexed is shown. In the figure, the horizontal axis indicates time and the vertical axis indicates transmission rate (bps). The data transmission rate of the mobile phone is, for example, 960.
0 bps.

In the example shown in FIG. 7
8000 bp compressed and encoded by the 29 encoding method
The s audio data 81 and the compression-encoded handwritten character image data 82 are multiplexed and transmitted. In this case, the handwritten character image data 82 is assigned a transmission rate of 1600 bps. Next, for example, when the image data is switched to the transmission target, the low-frequency component 83 of the compression-encoded image data is converted to single data at 9600 bps.
Is transmitted using the transmission rate.

When the transmission processing of the low-frequency component 83 of the image data is completed, the high-frequency component 85 of the compression-encoded image data is converted into the audio data 8 of 8000 bps.
4 and are sequentially transmitted. In this case, a transmission rate of 1600 bps is assigned to the high frequency component 85 of the image data. Then, when the transmission processing of the high frequency component 85 of the image data is completed, for example, 8000b
The ps audio data 86 and the 1600 bps handwritten character image data 87 are multiplexed and transmitted. It is assumed that the transmission rate assigned to each piece of media data is fixed.

As shown in FIGS. 5 and 6, the data communication apparatus 1 of the present embodiment transmits only one type of media data in accordance with the conditions set according to the type of media and the user's instruction. By combining the transmission processing method and the transmission processing method of multiplexing and transmitting two or more types of media data, a plurality of types of media data can be transmitted efficiently.

Next, a device that receives media data from the communication channel 27 will be described. The demodulator 31 has a function of receiving a signal transmitted from the transmission-side device through the communication path 27, demodulating the received signal into multiplexed data, and outputting the multiplexed data to the communication control unit 32. It has. Note that the demodulator 31 also receives, for example, encoded control data transmitted from the transmission side, and the encoded control data is input to the decoding unit 34 via the communication control unit 32 and the demultiplexing unit 32. .

The communication control unit 32 has a function of performing error correction decoding processing, data flow control, and the like.
It has a function of outputting data obtained by subjecting the multiplexed data input from 1 to error correction decoding processing or the like to the demultiplexing unit 33. Note that the communication control unit 32 may have a function of performing an error detection process or the like corresponding to, for example, the transmitting side.

The demultiplexing section 33 outputs a demultiplexing control section C35 for controlling the demultiplexing processing of the multiplexed data, a demultiplexer 36 for demultiplexing the multiplexed data, and outputs the media data to the decoding section 34. And a demultiplexing control unit D37. The demultiplexing control unit C35 has a function of controlling the demultiplexing process in the demultiplexing unit 33, and determines whether the data input from the communication control unit 32 is single data or multiplexed data. And if the data is single data, the single data is demultiplexed by the demultiplexing control unit D.
37, and has a function of outputting the multiplexed data to the separator 36 when the data is multiplexed data.

The demultiplexer 36 has a function of demultiplexing the multiplexed data input from the demultiplexing control section C35 into media data of each type and outputting the media data to the demultiplexing control section D37. The demultiplexing control unit D37 decodes the media data input from the demultiplexing control unit C35 and the media data of each type input from the demultiplexer 36 in accordance with each medium provided in the decoding unit 34. Parts 34a, 34b, 34
c is provided.

The decoding section 34 includes an image decoding section 34a for expanding and decoding the encoded image data, an audio decoding section 34b for expanding and decoding the encoded audio data, and expanding the encoded handwritten character image data. And a data decoding unit 34c for decoding. The image decoding unit 34a includes, for example, a plurality of types of decoding means for decompressing and decoding coded image data corresponding to the transmission side, and decodes coded image data to be decoded into a demultiplexing control unit. A function to input from D37,
It has a function of outputting decoded image data.
In this example, a plurality of types of decoding means for decompressing and decoding the encoded image data are composed of a plurality of types of decoders Q1 and Q
2, a decoder Q3,...

The audio decoding section 34b includes a plurality of types of decoding means for expanding and decoding the encoded audio data corresponding to, for example, the transmitting side, and demultiplexes the encoded audio data to be decoded. A function input from the conversion control unit D37,
It has a function of outputting decoded audio data.
In this example, a plurality of types of decoding means for decompressing and decoding coded audio data include a plurality of types of decoders T1 and T
2, the decoder T3,...

The data decoding section 34c includes a plurality of types of decoding means for decompressing and decoding encoded handwritten character image data, for example, corresponding to the transmitting side. It has a function of inputting character image data from the demultiplexing control unit 37 and a function of outputting decoded handwritten character image data. In this example, a plurality of types of decoding means for decompressing and decoding encoded handwritten character image data include a plurality of types of decoders J1, J2, J3,.
・ ・ It is composed of

The above-mentioned decoding unit 34 is provided with a demultiplexing unit 33
Based on the content of the encoding control data input via
Which encoding means (encoding method) is used for the encoded image data, encoded audio data, and encoded handwritten character image data input to the encoding units 34a, 34b, and 34c for each medium.
And has a function of determining whether the data is encoded by using the above. In addition, the decoding unit 34a, 34b, 34c provided for each media data is controlled, and the determination is performed for each media data. It has a function of decoding each media data by a decoding method corresponding to the encoding method. With this function, the receiving side can decode the encoded data for each medium received from the transmitting side by a decoding method corresponding to the encoding method.

With the above configuration, the receiving apparatus receives single data and multiplexed data transmitted from the transmitting apparatus to the communication path 27 via the communication path 27, and receives the single data. Can decode the data by a decoding process, separate the received multiplexed data into data for each medium, and decode the separated encoded data for each medium.

As described above, in the data communication device 1 of the present embodiment, the conditions set according to the type of media,
According to the user's instruction, the type of media data to be transmitted can be switched.In addition, according to the communication status and the type of the media, or an encoding unit suitable for compression-encoding the media data according to the user's instruction, for example, It is possible to dynamically select and switch during communication, thereby enabling communication of a plurality of types of media data even when a communication path with low communication quality is used, for example, in mobile communication or videophone. It is possible to realize efficient operation.

In the data communication apparatus 1, the conditions set according to the type of media serving as the basis for switching the type of media data to be transmitted from the input media data are the same as those described in the above embodiment. The present invention is not limited to this, and any condition may be set. Similarly, in the data communication device 1, the user's instruction serving as a basis for switching the type of media data to be transmitted from the input media data is not limited to the one shown in the above-described embodiment. An instruction may be given.

As another example of the condition set according to the type of media described above, for example, when transmitting a plurality of types of media data including audio data by a data communication device, the immediacy is ensured especially for audio data. When there is a request to do so, for example, normally, voice data is transmitted, and based on the fact that the input level of the voice detected by the voice detecting means of the data input unit 11 is equal to or lower than a predetermined threshold, Switching of another type of media data to the transmission target can also be set. In such a setting, another type of media data can be transmitted using a silent section of the voice, for example, after the transmission process of the other type of media data is completed, or when the input level is higher than a predetermined threshold. For example, when the voice is input, the voice data can be switched to the transmission target again.

Here, in the data communication apparatus shown in the above embodiment, the type of the media data to be transmitted is executed by the processor executing a control program in a hardware resource including a processor and a memory. However, in the present invention, each functional unit for executing these processes may be configured as an independent hardware circuit. Further, the present invention can be understood as a computer-readable storage medium such as a floppy disk or a CD-ROM storing the above-mentioned control program, and the control program is input from the storage medium to the computer and executed by the processor. Thereby, the processing according to the present invention can be performed.

Further, the data communication apparatus shown in the above embodiment has a plurality of types of encoding means for all of image data, audio data, and handwritten character image data to be transmitted. Although a configuration has been shown in which an encoding unit is selected from a plurality of types of encoding units in accordance with communication conditions and the like, for example, one type or two types of media data to be transmitted may be used instead of all types of media data to be transmitted. A configuration may be used in which a plurality of types of encoding units are provided only for some types of media data such as types and the encoding unit selection process is performed. In this case, for media data for which no encoding unit selection process is performed, for example, one or more types of encoding units may be provided, or no encoding unit is provided. Is also good. Further, the number of types of encoding means provided for the media data on which the selection processing of the encoding means is performed may be any number as long as it is two or more, and any type of encoding means may be used. Means may be used.

Further, in the data communication apparatus shown in the above embodiment, the function means for switching the type of the media data to be transmitted according to the condition set according to the type of the media, and the media data to be transmitted according to the instruction of the user. Although the configuration including both of the switching function units has been described, a configuration in which only one of these function units is included in the data communication device may be used.

Further, in the data communication apparatus shown in the above embodiment, the function means for selecting the coding means according to the communication status and the type of the medium, and the function means for selecting the coding means according to the instruction of the user. Although the case where both are provided is shown, a configuration in which only one of these functional units is provided in the data communication device may be used. Further, in the above-described embodiment, as a preferred embodiment, both a function unit for switching the type of media data to be transmitted to the data communication apparatus and a function unit for selecting a unit for compressing and encoding the media data are provided, and a plurality of types of media are Although the configuration for more reliably improving the data communication efficiency has been described, the data communication device does not necessarily have to have a function unit for selecting a unit for compressing and encoding the media data.

Further, in the data communication apparatus shown in the above embodiment, the case where image data, audio data, and handwritten character image data are transmitted as a plurality of types of media data has been described. The present invention is not limited to these media, and various media data such as text data may be used. Also, the number of types of media data of a plurality of types to be transmitted is not necessarily limited to the three types shown in the above-described embodiment.
The number of types of media data of a plurality of types may be arbitrary.

[0112]

As described above, according to the data communication apparatus of the present invention, when transmitting a plurality of types of media data by using the multiplexing process, the data is input according to the conditions set according to the type of the media. Since the type of media data to be transmitted is switched from the media data that has been transmitted, for example, it is possible to efficiently transmit a plurality of types of media data while ensuring the immediacy required for each type of media data. be able to.

Further, according to the data communication apparatus of the present invention, when transmitting a plurality of types of media data using the multiplexing process, the type of media data to be transmitted from among the input media data in accordance with a user's instruction. In order to switch, for example, according to the instruction input by the user in a timely manner according to the communication status and the like that fluctuates during communication,
It is possible to switch the type of media to be transmitted, thereby realizing efficient transmission of a plurality of types of media data.

Further, according to the data communication apparatus of the present invention, the type of media to be transmitted is switched according to the condition set according to the type of media as described above or the instruction of the user, and a plurality of types of media are transmitted. In transmitting the data, a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data are provided, and the communication status and the type of the medium are selected from the plurality of types of encoding means. Since the media data is encoded by the encoding means selected in response, for example, means suitable for encoding the media data can be dynamically selected and switched during communication, whereby It is possible to realize efficient transmission of various types of media data.

Further, according to the data communication apparatus of the present invention, the type of media to be transmitted is switched according to the conditions set according to the type of media as described above or the user's instruction, and a plurality of types of media are transmitted. When transmitting the data, a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data are provided, and a plurality of types of encoding means are selected from the plurality of types of encoding means in accordance with a user instruction. Since the encoding unit encodes the media data, it is possible to dynamically select and switch a unit suitable for encoding the media data during communication, for example, according to a user's instruction. Accordingly, it is possible to efficiently transmit a plurality of types of media data.

Further, according to the data communication apparatus of the present invention, when any one of the above-described data communication apparatuses transmits audio data and other types of media data as a plurality of types of media data, further inputting is performed. Detecting the input level of the input voice, and switching the other type of media data to a transmission target based on the input level being equal to or lower than a predetermined threshold, thereby using the silent section of the input voice. Since other types of media data are transmitted, it is possible to efficiently transmit a plurality of types of media data including the audio while ensuring the immediacy of the audio.

Further, according to the data communication device of the present invention, when transmitting any of a plurality of types of media data including voice data and handwritten character image data by any of the data communication devices described above, When voice data and handwritten character image data are input, by switching both the input voice data and handwritten character image data to the transmission target, both data are multiplexed and transmitted. It is possible to realize efficient transmission of a plurality of types of media data including data of both media while guaranteeing the synchronism between the voice and the handwritten character image.

[Brief description of the drawings]

FIG. 1 is a configuration example of a data communication device according to an embodiment of the present invention.

FIG. 2 is a diagram for describing a data conversion unit and a communication control unit in detail.

FIG. 3 is a diagram for explaining a process of switching the type of media data to be transmitted;

FIG. 4 is a diagram illustrating a process of switching the type of media data to be transmitted.

FIG. 5 is an example of a temporal change of a transmission rate.

FIG. 6 is an example of a temporal change of a transmission rate.

FIG. 7 is a configuration example of a data communication device according to a conventional example.

FIG. 8 is an example of a temporal change of a transmission rate according to a conventional example.

[Explanation of symbols]

1. Data communication device, 2. Input device, 2a ...
Digital camera, 2b, microphone, 2c, handwriting input screen, 11 data input unit, 12 data output unit, 13 data conversion unit, 14 communication control unit
15 Data operation unit, 16 Data temporary storage area, 21 Encoder, P1, P2, P3 Encoder, S1, S2, S3 Encoder, H1, H2, H
3 ··· Encoder, 22 ·· multiplexing unit, 23 ··· modulator, 24 ··· multiplexing control unit A, 25 ··· multiplexer,
26 multiplex control section B, 27 communication channel, 41
Pen for input, 61 ... Send button, 62 ... Receive button, 63 ... Mode button, 64 ... Type button,
65 status display area 66 pop-up menu

Continuation of the front page (72) Inventor Oichi Oyamada 3-14-20 Higashinakano, Nakano-ku, Tokyo Inside Kokusai Electric Corporation

Claims (6)

[Claims] 1. A data communication device for multiplexing and transmitting a plurality of types of media data, comprising: input means for inputting media data; and input media data selected from media data input according to a condition set according to the type of media. Switching means for switching the type of media data to be transmitted; multiplexing means for multiplexing two or more types of media data to be transmitted by the switching means; and transmission means for transmitting the media data. Characteristic data communication device. 2. A data communication apparatus for multiplexing and transmitting a plurality of types of media data, comprising: an input unit for inputting media data; and a type of media data to be transmitted from the input media data according to a user instruction. A data communication apparatus comprising: a switching unit that switches; a multiplexing unit that multiplexes two or more types of media data to be transmitted by the switching unit; and a transmission unit that transmits the media data. 3. The data communication apparatus according to claim 1, further comprising a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data. A data communication apparatus comprising: an encoding control unit that encodes the media data by an encoding unit selected according to a communication situation and a type of a medium from among the encoding units. 4. The data communication apparatus according to claim 1, further comprising a plurality of types of encoding means for compressing and encoding the media data with respect to at least one type of media data. A data communication device comprising: an encoding control unit that encodes the media data by an encoding unit selected according to a user's instruction from among the encoding units. 5. The data communication device according to claim 1, wherein the plurality of types of media data are audio data and another type of media data. A voice detection unit that detects an input level of the input voice, wherein the switching unit transmits the other type of media data based on an input level detected by the voice detection unit being equal to or less than a predetermined threshold. A data communication device, wherein the other type of media data is transmitted by the transmission unit using a silent section of the input voice by switching to a target. 6. The data communication apparatus according to claim 1, wherein the plurality of types of media data include audio data and handwritten character image data, and When voice data and handwritten character image data are input, the switching unit switches the input voice data and handwritten character image data together to the transmission target, and both data are multiplexed by the multiplexing unit. A data communication device for transmitting the data by the transmitting means.