JP2002123273A - Information terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information terminal which is provided with a sound reproduction function for music or the like and a image display function and can sufficiently exhibit the image display function according to user's needs while reproducing the sound satisfactorily. SOLUTION: A sound data reproduction processing part 11 realizes the reproduction function of sound data, such as music, and the image display function is realized by an LCD controller 13 and an LCD 14. During the reproduction of sound data, image data are displayed on the LCD 14. The user can select the image data to be associated with reproduced sound data through a key operation part 34. Association data which associate the sound data responding to an input operation of this user with image data are recorded on a flash memory 24. In the reproduction of the sound data, an image based on the associated image data is displayed on the display screen of the LCD 14 by referring to the associated data of the flash memory 24.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information terminal device having an audio display function for reproducing audio data such as music (music) and making it audible, and also having an image display function.

[0002]

2. Description of the Related Art With the spread of personal computers and the realization of anti-duplication technology and duplication control technology, which are technologies for protecting the copyright of digitized audio data (music data) such as music, music data can be reproduced. Music distribution services for distribution through communication networks are being provided.

In this music distribution service, a personal computer having a communication function is used to access, via a communication network, a server device on the side of a music data provider provided on the communication network, and store and hold the music data in the server device. It is intended to receive the distribution of the target music data from the set music data.

[0004] In this case, when the distribution of the music data is received, a charge is generated. Since the payment for the charge can be made by a credit card or withdrawal from a bank account, the user (user) can use the CD each time. There is an advantage that the user can purchase and use the desired music data whenever he likes, without having to go to the shop.

[0005] In addition to reproducing and listening to the distributed music data using a personal computer, the distributed music data can be reproduced using an MD (mini disc).
It is recorded on a so-called memory card or the like, which is a removable storage medium using a small magneto-optical disk or a semiconductor element, and reproduced and used by an MD player or a memory card player. It is becoming.

[0006] In this way, an environment is provided in which the server device on the music provider side is accessed at any time and the desired music data is downloaded from the server device and reproduced and listened to. Is coming.

[0007]

By the way, the transmission rate in the portable telephone network is considered to be improved in the future, and more data can be transmitted at high speed. For this reason, it has been considered that desired music data is downloaded from a server device on the provider side and used by using a mobile phone terminal or a portable communication terminal. In this case, a mobile phone terminal or a portable communication terminal is provided with a music data playback function, and a more convenient information terminal device can be realized in which a device for downloading music data and a device for playing back the music data are integrated. ing.

Some mobile phone terminals and mobile communication terminals have already been able to access so-called Web server devices provided on the WWW (World Wide Web) and browse contents such as Web pages. What has been provided has also been provided.

[0009] Some portable telephone terminals and portable communication terminals that can access the Web server device include a full-color LCD (Liquid Crystal) having a relatively large display screen for browsing contents such as Web pages.
al Display) has been provided, and a still image or the like can be displayed.

However, as described above, when the distribution of the target music data is performed using the mobile telephone terminal or the mobile communication terminal, and the music data is reproduced and listened to, the comparison is provided. No effective use of a full-color LCD having a large display screen has been considered.

Currently used portable MD (mini-disc) players, CD (compact disc) players, and the like can display at most a song name and a playing time because of their small display capability. It is about. However, simply displaying such information on a full-color LCD having a relatively large display screen of a mobile phone terminal or a mobile communication terminal does not fully utilize the image display function. It cannot be said that it provides a variety of image information that meets the needs of the user.

Therefore, when music data is provided from a music data provider, image data such as a still image such as a photograph or a moving image is provided in association with the music data to be provided. During reproduction of the music data, it is conceivable to display an image based on the provided image data on the LCD.

However, in this case, even if the music data is requested by the user, the image data does not meet the request of the user. For this reason, the image displayed by the image data provided by the music data provider may be unpleasant for the user, or may be annoying to the user.

[0014] For this reason, in an information terminal device such as a music playback device having a music playback function and an image display function as well as a mobile phone terminal or a mobile communication terminal, image information according to the user's needs can be obtained even when music is played. There is a demand for realization of a new information use form that enables users to effectively use both audio and video.

In view of the above, it is possible to provide a sound reproducing function of sound data such as music and an image displaying function to reproduce sound well and to provide image information according to the needs of the user. It is an object to provide an information terminal device.

[0016]

According to an aspect of the present invention, there is provided an information terminal apparatus comprising: a sound data reproducing unit for forming a reproduced sound signal from sound data such as music; An output end of the reproduced audio signal from the reproducing unit, an image data reproducing unit for forming a reproduced image signal from the image data, and a supply of the reproduced image signal from the image data reproducing unit. A display element for displaying a corresponding image, a receiving unit for receiving an instruction input for associating the audio data with the image data, and the audio data and the image data according to the instruction input received through the receiving unit. Storage means for storing association information for associating
When reproducing the audio data, the image data associated with the audio data is obtained based on the association information stored in the storage unit, and supplied to the image data reproduction unit. And display control means for displaying an image based on the image data on the display element.

According to the information terminal device of the first aspect, the receiving means receives an instruction input for associating a set of audio data such as music with a set of image data. The association information corresponding to the instruction input is stored in the storage unit. When audio data such as music is reproduced by the audio data reproducing unit, image data associated with the reproduced audio data is specified based on the association information stored in the storage unit. .

The specified image data is obtained from the location under the control of the display control means, supplied to the image data reproducing means, and an image based on the image data is displayed on the display element. Thus, when audio data such as music is reproduced, image data associated with the reproduced audio data is reproduced based on the association information corresponding to the instruction input of the user (user). You.

Thus, at the time of reproducing audio data such as music, the user can display an image based on the image data associated with the audio data. Since the displayed image is associated with audio data such as music by the user, the displayed image is not disturbed and the user can enjoy the displayed image together with the audio such as music to be reproduced. You can do so. Therefore, it is possible to effectively utilize the display element even at the time of reproducing audio data such as music, and realize an unprecedented use form of the information terminal device.

Further, the information terminal device according to claim 2 is:
2. The information terminal device according to claim 1, further comprising a built-in storage unit that stores the image data, wherein the display control unit obtains the image data from the built-in storage unit.

According to the information terminal device of the present invention, the image data associated with the music data or the image data associated with the music data is stored in a built-in storage such as a memory built in the information terminal device. It is stored in the means. Then, based on the association information stored in the storage unit, the display control unit reads out the target image data from the built-in storage unit, supplies the read image data to the image data reproduction unit, and displays the image based on the image data. It can be displayed on the device.

Thus, the image based on the image data corresponding to the audio data selected by the user is displayed on the display element using the image data stored in the built-in storage means, and is reproduced together with the reproduced sound. An image based on image data associated with audio data can also be displayed and enjoyed. Further, by updating the image data stored in the internal storage means, it is possible to easily arrange and increase the image data associated with the audio data.

The information terminal device according to claim 3 is
2. The information terminal device according to claim 1, further comprising: a connection end of a detachable storage unit that stores and holds the image data, wherein the display control unit is configured to perform the operation from the detachable storage unit connected to the connection end. It is characterized by obtaining image data.

According to the information terminal device of the third aspect, the image data associated with the music data or the image data associated with the music data is connected to the connection terminal provided in the information terminal device. It is stored and held in the removable storage means. The display control means reads out the target image data from the removable memory connected to the connection end of the removable storage means, supplies the read image data to the image data reproducing means, and outputs the image based on the image data to the display element. Be able to be displayed.

Thus, the image based on the image data corresponding to the audio data selected by the user is displayed on the display element using the image data stored in the removable storage means, and is reproduced together with the reproduced sound. It is also possible to display and enjoy an image based on image data associated with audio data.

Also, by updating the image data stored in the removable storage means, it is possible to easily arrange and increase the image data associated with the audio data. In addition, by preparing a plurality of detachable storage units storing different image data, more image data can be used more easily.

The information terminal device according to claim 4 is
4. The information terminal device according to claim 2, further comprising communication means for performing communication, obtaining the image data through the communication means, and storing and storing the image data. I do.

According to the information terminal device of the present invention, image data to be stored and held in the built-in storage means or the removable storage means can be obtained through the communication means. . Thereby, a large number of image data according to the needs of the user can be used in the information terminal device.

[0029] The information terminal device according to claim 5 is
2. The information terminal device according to claim 1, further comprising a communication unit for performing communication, wherein the display control unit obtains the image data through the communication unit.

According to the information terminal device of the fifth aspect, the image data associated with the music data or the image data associated with the music data is stored in an external server device. Then, based on the association information stored in the storage unit, the display control unit accesses the target server device through the communication unit to obtain the target image data,
This is supplied to the image data reproducing means so that an image based on the image data can be displayed on the display element.

Thus, the image data corresponding to the audio data selected by the user is displayed on the display element, and the image corresponding to the reproduced audio data is displayed and enjoyed together with the reproduced audio. Will be able to. In addition, the user can use image data that suits his or her preference from among a large number of image data that are abundantly prepared in the server device.

Further, the information terminal device according to claim 6 is
The information terminal device according to claim 4, wherein the communication unit is connected to a wireless network, and obtains the image data from a server device on the wireless communication network.

According to the information terminal device of the present invention, connection to a wireless network such as a cellular phone network is enabled by the communication means, and image data is provided through the wireless network. Will be able to. Thus, the image data is provided through the wireless network, and can be reproduced and used in the information terminal device.

Further, the information terminal device according to claim 7 is
6. The information terminal device according to claim 4, wherein the communication unit is connected to a wired network and obtains the image data from a server device on the wireless communication network.

According to the information terminal device of the present invention, connection to a wired network such as a public telephone network is enabled by the communication means, and image data can be provided through the wired network. Will be able to. As a result, the image data is provided through the wired network, and can be reproduced and used in the information terminal device.

Further, the information terminal device according to claim 8 is
6. The information terminal device according to claim 4, wherein the communication unit obtains the image data from a device holding the image data by a short-range wireless communication method.

According to the information terminal device of the present invention, the communication means enables short-distance wireless communication by a short-distance wireless communication system such as Bluetooth (Bluetooth). Image data can be provided from a device having a function as a server device.

Thus, the image data is provided by the short-range wireless communication, and can be reproduced and used in the information terminal device. Further, the communication means may be provided in the removable storage means.

Further, the information terminal device according to claim 9 is
6. The information terminal device according to claim 4, wherein the communication unit obtains the image data from a device holding the image data through a wired digital interface.

According to the information terminal device of the ninth aspect, the IEEE (Institute of Electron)
Ial Electronics Engineer
s) 1394 digital interface and USB
Through an (Universal Serial Bus) standard digital interface, image data can be provided from a device that functions as a server device such as a personal computer.

Thus, the image data is provided through the wired digital interface, and can be reproduced and used in the information terminal device.

The information terminal device according to a tenth aspect is the information terminal device according to the first aspect, wherein the association information stored in the storage means includes a storage destination of the image data. The display control means obtains the target image data from the storage location indicated by the information indicating the storage location added to the association information.

According to the information terminal device of the tenth aspect, the information indicating the storage location of the image data associated with the audio data is added to the association information stored in the storage means. The information indicating the storage location of the image data is information indicating a built-in storage unit of the information terminal device, a detachable storage unit, a server device provided on a communication network such as the Internet, and the like. The display control means acquires target image data from a possible location indicated by the information indicating the storage location of the image data, and displays an image corresponding to the image data on the display means.

Thus, even if there are a plurality of locations where the image data is stored, the desired image data can be obtained and used. Therefore, the image data can be distributed and stored in a plurality of storage locations, and more image data can be used.

[0045]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an information terminal device according to the present invention will be described below with reference to the drawings. In the embodiment described below, a case where the information terminal device according to the present invention is applied to a mobile phone terminal will be described as an example.

[Configuration of Mobile Phone Terminal] FIG. 1 is a block diagram for explaining a mobile phone terminal according to this embodiment. As shown in FIG. 1, the mobile phone terminal according to the present embodiment includes a transmitting / receiving antenna 1, an antenna duplexer 2, a receiving unit 3, a baseband processing unit 4, and a DSP used as a voice codec as parts for realizing a communication function. (Digi
tal Signal Processor 5, a speaker (receiver) 6, a microphone (transmitter) 7, a transmitting unit 8, and a frequency synthesizer 9.

As a part for realizing a reproduction function of audio data such as music (music), an audio data reproduction processing unit 1
1, an audio signal output terminal 12 is provided. Further, the mobile phone terminal according to the present embodiment displays various types of image information such as characters, graphics, symbols, still images, and moving images.
CD (Liquid Crystal Display)
A controller 13 and an LCD 14 are provided.

The LCD 14 is a TFT color liquid crystal display (Thin Fi) having a relatively large display screen.
lm Ttansistor Color Liqui
(Crystal Display). And
The mobile phone terminal according to the present embodiment is connectable to the Internet, and also displays image information such as a so-called Web page and e-mail provided on the Internet on the LCD 14 so that the displayed image can be viewed well. To be able to.

Further, the mobile phone terminal of this embodiment is equipped with a camera unit 15 and has a function as a so-called digital camera. Thus, an image such as a landscape can be recorded as image data and displayed on the LCD 14 for confirmation when going out.

Further, the mobile phone terminal of this embodiment
A ringer 16 is provided for emitting a ring tone for notifying an incoming call, emitting a warning sound, and the like, and also has a vibrator (not shown) for notifying an incoming call. Further, as shown in FIG. 1, the mobile phone terminal of this embodiment is provided with a control unit 20 for controlling each unit of the mobile phone terminal.

As shown in FIG. 1, the control unit 20 has a CPU
(Central Processing Unit)
21 and ROM (Read Only Memory)
22 and SRAM (Static Random Acc)
ess Memory) 23 and flash memory 24
Are microcomputers formed by being connected through the CPU bus 25.

Here, the ROM 22 stores in advance a program to be executed by the CPU 21 and necessary data such as display fonts. Also, SR
The AM 23 is mainly used as a work area, and stores data in the middle of calculation as needed while the CPU 21 is executing a program.
Is used to temporarily store data exchanged with each unit.

The flash memory 24 has an EEPROM
OM (Electrically Erasable a
nd Programmable ROM). Even when the power of the mobile phone terminal is turned off, for example, the immediately preceding setting conditions are stored, and when the usage method is set such that the same setting is performed at the next power-on time, these are used. Are stored. That is, the flash memory 24 is a non-volatile memory in which the data stored therein does not disappear even if the power of the mobile phone terminal is turned off.

The control unit 20 has an external memory interface (hereinafter abbreviated as an external memory I / F).
An external memory slot 32 is connected through a key interface 31 and a key interface (hereinafter referred to as a key I / F).
Abbreviated. ) 33, a key operation unit 34 is connected.

The external memory slot 34 is provided with a detachable opening of the detachable external memory 40, reads data from the attached external memory 40 and supplies it to the control unit 20, and transmits data from the control unit 20 to the external memory 40. It has a function as a so-called reader / writer for recording in the memory 40. Here, the external memory 40 is a so-called memory card such as a memory stick manufactured by Sony Corporation, for example, and is a removable storage medium using a semiconductor memory.

The key operation section 32 has a plurality of operation keys such as a dial key (numeric keypad) and various function keys, and various operation keys from the user are transmitted through the operation keys of the key operation section 32. Instruction input can be received. Control unit 2
Numeral 0 indicates that data can be transmitted / received to / from an external device such as a personal computer through an external interface (hereinafter abbreviated as an external I / F) 35 and an input / output terminal 36. I have.

[Operation of Reception System of Mobile Phone Terminal] Next, the reception system of the mobile phone terminal of this embodiment will be described. The received signal received through the antenna 1 is supplied to the receiving unit 3 through the antenna duplexer 2. The receiving unit 3
The necessary band limitation and AGC (Automatic Gain Cont
), and based on the signal from the frequency synthesizer (local oscillator) 9, performs processing such as setting the frequency of the received signal to a constant frequency, and supplies the processed signal to the baseband processing unit 4. I do.

The baseband processing unit 4 performs A / D conversion on the signal from the receiving unit 3 to remove effects such as fading,
It determines the type of the received signal, performs de-interleaving, and performs error correction, performs appropriate decoding processing, and separates audio data from other communication data. Then, the audio data is supplied to the DSP 5 and is other communication data.
For example, various control information and character data are stored in the control unit 2.
0 is supplied.

The DSP 5 D / A converts the audio data from the baseband processing unit 4 to form an analog audio signal, and supplies the analog audio signal to the speaker 6. The speaker 6 is DS
Driven by the analog sound signal from P5, sound according to the received signal is emitted from the speaker 6.

On the other hand, from the baseband processing unit 4 to the control unit 2
0 is the control data or character data for the mobile phone terminal, the S
The data is temporarily stored in the RAM 23 and used in this mobile phone terminal. The other communication data is recorded in the detachable external memory 40 mounted in the external memory slot 32 via the external memory I / F 31 or the external I / F 35 and the input / output terminal 36, for example. Through a personal computer connected to the mobile phone terminal.

[Operation of Transmission System of Mobile Phone Terminal] Next, the transmission system of the mobile phone terminal of this embodiment will be described. The microphone 7 converts the collected voice into an analog voice signal and supplies the analog voice signal to the DSP 5. DSP5
The analog audio signal from the microphone 7 is A / D converted to form a digital audio signal, which is supplied to the baseband processing unit 4.

[0062] The baseband processing section 4 encodes the digital audio signal from the DSP 5 by a predetermined coding method, compresses it, and combines it into a predetermined block. Further, the baseband processing unit 4 combines digital data supplied to the mobile phone terminal through the external input / output terminal 36 and the external I / F 35 into predetermined blocks. The baseband processing unit 4 combines the compressed digital audio signal and digital data from an external device, and supplies the digital data to the transmission unit 8.

The transmission section 8 forms a modulation signal from the digital data from the baseband processing section 4 and converts the modulation signal into a predetermined transmission frequency and a modulation signal from the frequency synthesizer 9. The signal is mixed to form a modulated signal for transmission. The modulation signal for transmission formed in the transmission unit 8 passes through the antenna duplexer 2 and
It is transmitted from the transmitting / receiving antenna 1.

In the portable telephone terminal of this embodiment having such a receiving system and a transmitting system, at the time of standby reception, the control unit 20 monitors the received signal from the baseband processing unit 4 by monitoring the received signal. Detects an incoming call to its own device. Then, when detecting an incoming call to the own device, the control unit 20 controls the ringer 16 to emit a ringing sound (ringer sound) so that the incoming call to the own device is transmitted to the mobile phone terminal. Notify the user.

When the user of the mobile phone terminal responds to the incoming call by performing an off-hook operation such as pressing an off-hook key provided on the key operation unit 34, the control unit 20 transmits Through the system, connect the communication line by sending a connection response, etc., as described above,
Calls can be made by the operations of the receiving system and the transmitting system.

When a call is made from the portable telephone terminal of this embodiment, after performing an off-hook operation such as pressing the off-hook key of the key operation unit 32, the dial key of the key operation unit 32 is used. The dialing operation is performed by selecting the telephone number of the other party from a telephone number list registered in advance.

Thus, the control unit 20 forms a call request and transmits the call request through the transmission system, thereby connecting the communication line with the intended destination telephone terminal. Then, an incoming response from the other party is returned,
When the connection of the communication line is confirmed, a call can be made by the operation of the receiving system and the transmitting system as described above.

As described above, the portable telephone terminal according to the present embodiment is capable of connecting to the Internet, and through a provider function provided by a telephone company.
Sends and receives e-mails and URLs (Uniform
m Resource Locators) to obtain and send information such as a web page,
It can be displayed on the LCD 14, recorded on the external memory 40 through the external memory I / F 31 and the external memory slot 32, and output to an external device through the external interface 35 and the input / output terminal 36. .

[Digital Camera Function] Next, the digital camera function of the portable telephone terminal of this embodiment will be described. As shown in FIG. 1, the mobile phone terminal of this embodiment includes a camera unit 15. The camera unit 15 includes an objective lens and a CCD (Charge C).
(e.g., coupled device).

Then, the image at the time when the shutter provided on the mobile phone terminal of this embodiment is pressed is displayed as the CC.
D, converts it into an electric signal, and converts it into image data (digital data). In this embodiment, the external memory 4 inserted in the external memory slot 32 is used.
0 can be recorded.

The external memory 40 is a recording medium for storing application data, as will be described later, and can store and hold audio data such as music (music data) and image data. is there. The external memory 40 has a storage capacity of, for example, tens to hundreds of megabytes or more. In the case of a still image, the data amount is, for example, 40 to 50 kilobytes when the image is displayed in 16 colors and 190 dots × 100 dots, so that a large number of images can be stored. .

When the image data reproduction key of the key operation unit 34 of the portable telephone terminal is operated, this is notified to the control unit 20. When receiving the instruction to reproduce the image data through the key operation unit 34, the control unit 20 reads out the image data recorded in the external memory 40 inserted in the external memory slot 32, and sends this to the LCD controller 13. Supply.

As a result, the LCD controller 32
An image corresponding to the image data from the control unit 20 is displayed on the LCD 14.
Is displayed on the display screen. As described above, the mobile phone terminal according to the present embodiment has a digital camera function, can capture an image at a place where the user is out, and can store the captured image as digital data in the external memory 40. Play back the captured image as necessary to perform LC
It is displayed on the display screen of D14 and can be viewed.

[Reproduction of audio data such as music]
Next, an audio reproduction function for reproducing a set of audio data (digital data) such as music provided in the mobile phone terminal of the present embodiment will be described. Here, a case will be described as an example where audio data such as music recorded in an external memory 40 inserted into an external memory slot 32 of the mobile phone terminal of the present embodiment is reproduced. In the case of audio data such as music, the amount of data depends on the compression method.
The voice data for one minute is about 5 megabytes, and a plurality of voice data can be stored in the external memory.

The external memory 40 detachably attached to the external memory slot 32 of the portable telephone terminal of this embodiment
It is mainly used as a recording medium for storing data of an application, and as described above, accesses a Web server provided on the Internet, and downloads audio data of music (downloaded from the Web server). Music data), image data such as still image data and moving image data.

Since the external memory 40 is a so-called removable medium that can be attached and detached, music data and image data are recorded using a personal computer, and are used in a portable telephone terminal. You can also. In the future, with the increase in storage capacity, like CDs (compact discs),
It is conceivable that an external memory in which audio data such as music is recorded is manufactured and distributed, but it is also possible to use such a distributed external memory.

The external memory 40 storing the music data is loaded into the external memory slot 32 by the above-described method, and the operation keys of the key operation unit 34 are operated, whereby the music reproduction instruction and Input an instruction to select music to play. These instructions are supplied from the key operation unit 34 to the control unit 20 through the key I / F 33.

In response to an instruction input from the user, the control unit 20 outputs audio data stored in the external memory 40 loaded in the external memory slot 32,
The audio data of the music designated by the user is read, and the read music data is supplied to the audio data reproduction processing unit 11.

The audio data recorded in the external memory is usually subjected to data compression by a predetermined data compression method, and the audio data reproduction processing unit 11 compresses and decompresses the audio data supplied thereto to perform data compression. The audio data is restored to the previous original audio data, and is converted from digital to analog to form an analog audio signal, which is output through the audio output terminal 12.

A headphone is connected to the audio output terminal 12. As a result, the sound based on the sound signal read from the external memory 40 and reproduced by the sound data reproduction processing unit 11 is emitted from the headphones connected to the sound output terminal 12, and the user can listen to the sound. Can be.

In this embodiment, the audio data reproduction processing section 11 includes an audio amplification circuit (audio amplifier), a sound quality adjustment circuit, and the like, and controls a control signal from the control section 20 in response to an instruction input from the user. Thus, the volume and the sound quality can be adjusted, and the adjusted audio signal can be output.

Further, in this embodiment, as shown in FIG. 1, a call is made using the receiver 6 and the transmitter 7, and reproduced sound such as music is connected to the sound output terminal 12. It is configured to listen using headphones. However, it is also possible to use headphones instead of the receiver 6 during a call.

In this case, a switch circuit (output selection circuit) for switching between outputting the audio signal from the codec 5 and outputting the audio signal from the audio data reproduction processing section 11 is provided at a stage preceding the audio output terminal 12. It can be realized by providing.

[Correlation between Audio Data such as Music and Image Data] FIG. 2 is a diagram for explaining the appearance of the portable telephone terminal of this embodiment, and FIG. FIG. 4 is a diagram for describing a display example of an image displayed on LCD 14 of the telephone terminal.

As shown in FIG. 2, in the portable telephone terminal of this embodiment, headphones 5 are connected to audio output terminal 12.
0 can reproduce audio data such as music recorded in the external memory 40 loaded in the external memory slot 32 and listen to audio through the headphones 50 in accordance with the audio data. It is.

Although not shown, the loading port of the external memory slot 32 into which the external memory 40 is loaded is provided in a portion that does not interfere with the operation by the user, such as the side surface of the mobile phone terminal of the present embodiment. .

As shown in FIG. 2, the portable telephone terminal of this embodiment has an LCD 14 capable of displaying a relatively large display screen in full color. In the case where audio data such as music recorded in the external memory 40 is reproduced using the mobile phone terminal of this embodiment, LC data is reproduced based on information recorded together with the audio data.
At D14, as shown in FIG. 3A, character information such as the title of the music to be played and the playing time can be displayed to notify the user.

However, simply displaying character information such as a title and performance time on the LCD 14 just when reproducing audio data such as music is not enough to provide the image display function of the portable telephone terminal of this embodiment. It cannot be said that the image information is utilized effectively, and it cannot be said that the image information is provided according to the needs of the user.

Therefore, in the portable telephone terminal according to the present embodiment, a set of audio data such as music reproduced by the portable telephone terminal and image data such as a still image and a moving image are transmitted to the portable telephone terminal. Correspondence is made according to the instruction of the terminal user.

When audio data such as music is reproduced, if there is image data associated with the reproduced audio data, for example, as shown in FIG. An image based on the associated image data is displayed on the LCD 14 so that audio such as music can be heard, and an image associated with the audio is displayed on the LCD 14 so that the image can be viewed. I have.

In this embodiment, audio data such as music and image data are stored in the external memory slot 32.
Is recorded and held in the external memory 40 loaded in the memory. In this case, the recording format of the external memory is
For example, MS-DOS (Microsoft Disk)
Operating System: an operating system manufactured by Microsoft Corporation in the United States), but the recording area is determined by the type of data.

For example, image data is recorded in a DCIM directory under the root directory, and audio data such as music is recorded in a VOICE directory under the root directory. Then, for each of the plurality of image data and the plurality of audio data, an 8-character A
It can be distinguished by the file name of the SCII code and the extension of three characters. Therefore, the target audio data or image data is specified by the file name or the file name and the extension, and the specified audio data or image data can be read and used.

For example, when audio data of a favorite music is downloaded via a network such as the Internet and recorded in the external memory 40, a scene matching the image of the downloaded music is taken by the camera unit 1.
Shoot through 5. Then, association information (combination information) in which audio data of the music is associated with photographed image data is created.

Then, the audio data of the music is reproduced,
When listening to a sound corresponding to the sound data, image data associated with the sound data of the music is specified based on the above-mentioned association information, and an image based on the specified image data is displayed on the LCD 14.

Thus, it is not only possible to simply play back and listen to music, but also to automatically reproduce an image associated with the music to be played back and listened to by the user of the portable telephone terminal as the listener. The image can be displayed on the LCD 14 and the image can be enjoyed. That is, in addition to the sound reproduction function of the mobile phone terminal of the present embodiment, the image display function can be effectively utilized, and an image according to the needs of the user can be provided together with the sound.

In this embodiment, the audio data such as music recorded in the external memory 40 is recorded in advance from a Web server on the Internet or a personal computer as described above. It is like that.

The image data recorded in the external memory 40 is, like audio data such as music, downloaded from a Web server on the Internet or recorded in advance using a personal computer. The user himself / herself takes a picture through the camera unit 15 of the mobile phone terminal of this embodiment.

In the portable telephone terminal of this embodiment, when reproducing audio data such as music recorded in the external memory inserted into the external memory slot 32, the reproduced audio data is Image data.

As described above, when the external memory in which audio data such as music is recorded is loaded in the external memory slot 32 of the portable telephone terminal of this embodiment,
Upon receiving a music playback instruction and an instruction to select music to be played through the key operation unit 34, the control unit 20 reads out the instructed audio data from the external memory and supplies it to the audio data reproduction processing unit 11, Play the selected audio data.

During reproduction of the audio data, the key operation unit 3
When the control unit 20 receives an instruction to select image data through the external memory 4, the control unit 20 reads out the image data recorded in the external memory 40 loaded in the external memory slot 32, and reads out the read image data through the LCD controller 13. An image corresponding to the data is displayed on the LCD 14.

In the portable telephone terminal of this embodiment, the image displayed on the LCD 14 can be changed every time a selection instruction of image data is received from the user through the key operation unit 34. . Then, when the target image is displayed on the LCD 14, the user operates a predetermined key provided on the key operation unit 34, thereby instructing the storage of the association information between the voice data and the image data (registration). Instructions).

When the control unit 20 receives this registration instruction, the control unit 20 forms association information for associating the audio data being reproduced with the selected image data, and records this in the flash memory 24. In this case, the association information includes a file name and extension of audio data and a file name and extension of image data, as described later.

Then, when reproducing the audio data such as music recorded in the external memory, the control unit 20
By referring to the association information recorded in the flash memory 24, if there is image data associated with the audio data to be reproduced, the image data is read from the external memory and supplied to the LCD controller 13. An image based on the read image data is displayed on the display screen of the LCD 14. Thereby, as described above, the sound based on the sound data instructed by the user and the image associated with the sound data are reproduced and provided to the user.

[Association Information] Next, the association information to be stored in the flash memory 24 of the portable telephone terminal of this embodiment will be described. FIG.
FIG. 3 is a diagram for explaining association information recorded in a flash memory 24 of the mobile phone terminal according to the embodiment.

As described above, in the portable telephone terminal of this embodiment, the audio data and the image data recorded in the external memory can be distinguished by the file name and the extension. Therefore, the control unit 20 of the mobile phone terminal according to the present embodiment creates association information in which the file name and extension of audio data are associated with the file name and extension of image data.

When audio data such as music recorded in the external memory 40 is reproduced, the file name and extension of the audio data being reproduced are recognized (understood) by the control unit 20. By performing a predetermined operation, the control unit 20 sequentially reads out the image data stored in the external memory 40 in accordance with a user's instruction, and supplies this to the LCD controller 13.
An image based on the read image data is displayed on the display screen of the LCD 14.

At this time, the user can sequentially change the image data to be read by performing a predetermined page change operation (input of an instruction to select image data). In this case, the control unit 20 recognizes the file name and extension of the image data read from the external memory 40. When the target image is displayed on the LCD 14, the user inputs a registration instruction through the key operation unit 34.

When receiving the input of the registration instruction from the user, the control unit 20 determines the file name and extension of the audio data of the audio being reproduced and the file name and extension of the image data of the image being displayed. The associated information is formed and stored in the flash memory 24.

In the case of the example shown in FIG. 4, the audio data having the file name “Audio A” has the file name “I
The image data having the file name “Image2” is associated with the audio data having the file name “AudioB”.
Similarly, the audio data having the file name “AudioC” includes image data having the file name “Image3”, the audio data having the file name “AudioD” includes the image data having the file name “Image4”, and The audio data having the file name “AudioE” is associated with image data having the file name “Image5”.

Based on the association information shown in FIG. 4, control unit 20 specifies the image data associated with the audio data instructed to be reproduced, reads the image data from the external memory, and reproduces the image data. Be able to be. Therefore, for example, the file name "Audio
In the case where the audio data having the file name “A” and the extension “aaa” is reproduced, the control unit 20 transmits the image data having the file name “Image1” and the extension “iii” associated with the audio data. Is read from the external memory and reproduced.

[Relationship Between Audio Data and Image Data] Next, referring to the flowchart of FIG.
A reproduction process of audio data and image data, and a process of associating audio data with image data, performed in the mobile phone terminal of this embodiment will be described. In the mobile phone terminal of this embodiment, when the power is turned on, the control unit 20 displays a normal screen on the LCD 14 so as to select an executable application (step S101). Here, the normal screen displays a list of executable applications and has a function as a menu screen for selecting an application to be executed.

When the control unit 20 receives an input of an instruction to execute a voice reproduction application (input of a music reproduction instruction) through the key operation unit 34 of the mobile phone terminal (step S102), the control unit 20 is loaded in the external memory slot 32. A list of audio data such as music recorded in the external memory 40 is displayed on the LCD 14, and a music selection instruction input from the user to instruct reproduction is received (step S102).

When the control unit 20 receives a music selection instruction input of music to be reproduced, it determines whether or not there is an image associated with the selected music from a plurality of pieces of association information in the flash memory 24. With reference to the association information table, it is determined whether there is image data associated with the selected (reproduced) audio data (step S104).

If it is determined in the determination processing of step S104 that there is image data associated with the audio data for which reproduction has been instructed, the control unit 20 associates the image data with the audio data for which reproduction has been instructed. The read image data is read from the external memory 40 (step S105), and a process of rewriting an image displayed on the display screen of the LCD 14 with an image based on the read image data to an image based on the newly read image data (step S10).
6).

In this embodiment, the audio data instructed to be reproduced is also read from the external memory 40, and reproduction of the read audio data is started (step S108). If it is determined in the determination processing of step S107 that there is no image data associated with the audio data for which reproduction has been instructed, the control unit 20 does not update the display data (step S107).
107), the process proceeds to step S108, and the reproduction process of the audio data instructed to be reproduced is started.

Then, the control unit 200 controls the external memory 40
It is determined whether or not the reproduction of the sound by the sound data has been completed based on the sound data from (step S11).
0). In the determination processing of step S110, when it is determined that the reproduction of the audio data whose reproduction has been instructed has ended, the control unit 20 ends the processing shown in FIG. 5, and waits for an instruction input from the user, for example.

If it is determined in step S110 that the reproduction of the audio data whose reproduction has been instructed has not been completed, it is determined whether an instruction to select image data to be associated with the audio data to be reproduced has been input. (Step S111).

If it is determined in step S111 that the input of the instruction to select the music data to be associated with the audio data to be reproduced has not been received, control unit 20 repeats the processing from step S109. In the determination processing of step S111, when it is determined that the input of the instruction to select the image data to be associated with the audio data to be reproduced is received, the control unit 20 transmits the association information between the audio data being reproduced and the image data selected this time. To accept a registration instruction indicating whether to register,
It is determined whether a registration instruction has been received (step S1).
12).

If it is determined in step S112 that the registration instruction has not been received, the control unit 20 repeats the processing from step S109, and can sequentially select the image data as described above. To do. In the determination processing of step S112,
When it is determined that the registration instruction has been received, the control unit 20
Determines whether the association information for the audio data being reproduced already exists (step S113).

In the determination processing of step S113, when it is determined that the association information for the audio data being reproduced already exists in the flash memory 24,
The control unit 20 rewrites the association information of the flash memory 24 by overwriting the previous association information of the audio data being reproduced with the new association information (step S114), and thereafter, the processing from step S109. repeat.

If it is determined in the determination processing of step S113 that the association information for the audio data being reproduced does not yet exist, new association information is registered (added) to the flash memory 24 ( Step S115), and thereafter, the processing from step S109 is repeated.

Immediately after Steps S114 and S115, an image based on the newly selected image data is displayed, so that the user can enjoy the sound of the reproduced audio data and the image. Is done.

As shown in the flowchart of FIG. 5, the mobile phone terminal of this embodiment, when instructed to reproduce audio data such as music, is associated with the instructed audio data. It is confirmed whether or not the image data exists, and if it exists, an image based on the image data is displayed on the display screen of the LCD 14.

Further, the image data to be reproduced can be associated with the audio data whose reproduction is instructed, even if there is no image data associated with the audio data. Even if the association information is registered, it can be changed, and if it has not been registered, it can be added.

As described above, in the portable telephone terminal of this embodiment, the audio data such as music can be associated with the image data, and when the reproduction of the audio data such as music is instructed, The image data associated with the audio data can be automatically displayed on the LCD 14.

As a result, it is possible to realize a new use form of audio data and image data that can provide image information according to the needs of the user when reproducing audio data such as music. Further, when reproducing audio data such as music, the image display function of the mobile phone terminal can be fully utilized.

[0127] In addition to displaying information such as the song title and the playing time, the user can also display his or her favorite images during music playback, so that the user can have more fun playing music.

Further, for example, the mobile phone terminal according to the present embodiment creates correspondence information between an image taken by a digital camera function when traveling and a song that was popular at that time or a song that was frequently heard. By doing so, when the song is reproduced, the image corresponding to the song is automatically displayed, so that the memories of the trip can be recalled more clearly.

Further, it is possible to provide a new creative work of searching for an image that matches the user's favorite music or photographing using a digital camera function and associating them with each other. The added value of the telephone terminal can be increased.

[Other Examples of Storage Location of Audio Data and Image Data] [Use of Built-in Memory] In the above-described embodiment, audio data such as music and image data are both
The description has been made assuming that the data is stored and held in the external memory. However, it is not limited to this. When a flash memory or the like, which is a built-in storage unit built in the mobile phone terminal, has a large storage capacity, the audio data and the image data are stored and stored in the built-in storage unit, and read out and reproduced therefrom. You can also do so.

As the built-in storage means, in addition to a semiconductor memory such as a flash memory, various storage media built in a portable telephone terminal such as a miniaturized disk device can be used. Alternatively, the audio data and the image data may be separately stored in a plurality of built-in memories.

[Sharing of Internal Memory and External Memory]
One of the audio data such as music and the image data may be stored in the built-in storage medium, and the other may be stored in the external memory. In addition, as a removable storage medium, in addition to an external memory such as a so-called memory card,
Various removable name storage media such as a floppy (registered trademark) disk and an MD (mini disk) may be used.

[Acquisition of Data by Communication] The mobile telephone terminal of the above-described embodiment has a communication function and can be connected to the Internet. For this reason, one or both of audio data such as music and image data may be supplied from a server device connected to a communication network such as the Internet.

In this case, the URL of the target server is transmitted from the portable telephone terminal of this embodiment, and the server receives distribution of audio data and image data. Voice data and image data from the server device are transmitted and received by the transmitting / receiving antenna 1, the antenna duplexer 2,
The signal is supplied to the control unit 20 through the receiving unit 3 and the baseband processing unit 4.

The control unit 20 reproduces and uses the audio data by supplying it to the audio data reproduction processing unit 11 and supplies the image data to the LCD controller 13. can do. In this case, the audio data and the image data do not need to exist in the same server device, but may each exist in different server devices.

One of the audio data and the image data may be stored in a built-in memory or an external memory of the portable telephone terminal, and the other may be obtained and used through a communication network such as the Internet. . Instead of directly using the audio data and image data from the server device, the data may be stored once in a built-in memory or an external memory of the mobile phone terminal, and may be used.

[0137] In addition, Bluetooth is added to the mobile phone terminal.
By installing a short-range wireless communication function such as
Of course, audio data and image data can be provided from devices such as a personal computer, an audio data and image data providing device, and a vending device in the vicinity.

In this case as well, audio data and image data are provided from nearby devices by short-range wireless communication, and are temporarily stored and used in a built-in memory or an external memory of the mobile phone terminal. Of course it is good.

The short-range wireless communication function such as Bluetooth is installed not in the portable telephone terminal but in, for example, an external memory which is a removable recording medium, and the short-range wireless communication function installed in the external memory is provided. , Audio data such as music and image data can be acquired and stored in an external memory, and can be used.

It is also possible to receive audio data and image data from devices such as a personal computer, a device for providing audio data and image data, and a vending device through a wired digital interface such as IEEE 1394 or USB. Also in this case, the audio data and the image data from the device connected by the digital interface may be stored in the built-in memory or the external memory of the mobile phone terminal and then used.

Further, by providing the portable telephone terminal with a function capable of connecting to a public telephone network drawn into a home such as a home, voice such as music can be directly connected to a wired network such as a public telephone network. Of course, distribution of data or image data may be received.

As described above, the storage destinations of the audio data and the image data include a built-in storage medium such as a built-in memory, a removable storage medium such as an external memory, a server device connectable through a wireless network such as a mobile phone network, and a public device. There are a server device connectable through a wireless network such as a telephone network, a device connectable by short-range wireless communication such as Bluetooth, and a device connectable through a wired digital interface.

Further, as in the above-mentioned cases (1) to (4), a storage section for voice data and image data can be provided outside the mobile phone terminal, so that the flash memory of the mobile phone terminal can associate the voice data with the image data. By storing the data, both the audio data and the image data can be provided and used from the outside, and the memory of the mobile phone terminal can be effectively used.

[Plural Use of Storage Locations of Audio Data and Image Data] As described above. Audio data such as music and image data can be stored in various places. For this reason, for example, audio data such as music is stored in an external memory, while image data is stored in various memories such as an internal memory, an external memory, a server device on a communication network, and devices connected to short-range wireless communication. Of course, you can store it in a place.

In this case, however, it is necessary to ensure that the portable telephone terminal can know where the image data associated with the audio data to be reproduced is stored. Therefore, in the case of this example, as shown in FIG. 6, information indicating the storage location of the image data is added to the association information.

Therefore, at the time of acquiring the image data, it is possible to designate from which storage location the image data is to be acquired, and when the image data is acquired from the designated storage location, the storage location is determined. Information indicating the location is added to the association information.

In the example of FIG. 6, the image data with the file name "Image1" is stored in the internal memory, and the image data with the file name "Image2" is stored in the external memory according to the information in the storage location column. Is shown. If the storage location is an external memory, identification information such as a sequence number of the external memory indicating which external memory is used is also added.

Also, In the case of the example of FIG. 6, the information in the storage location column indicates that the image data with the file name “Image3” is stored in a server device on the Internet, and the URL of the server device is stored in the storage location column. Is recorded. The file name is "Image4"
Indicates that the image data having the file name “Image5” is stored in the device connected by the digital interface.

As described above, since the portable telephone terminal clearly grasps the storage location of the image data, the image data stored anywhere can be quickly and reliably obtained and used. To be.

[0150] Here, the case where there are a plurality of storage locations for image data has been described as an example. However, information indicating the storage location is similarly added to audio data such as music, so that the purpose and the storage location are added. No matter where the audio data is located, it can be obtained and used.

In the above-described embodiment, the case where the information terminal device according to the present invention is applied to a portable telephone terminal has been described as an example, but the present invention is not limited to this. Information such as a music playback device having a built-in memory capable of storing audio data and image data, or an image display function capable of connecting an external memory capable of storing audio data and image data The present invention can be applied to a terminal device.

Also, a PDA (Personal Dig)
The information terminal device according to the present invention can be applied to a personal information terminal called an italian assistant, a personal information terminal having a communication function, and an information terminal device such as a personal computer. In particular, since the mobile phone terminal has a communication function and a relatively advanced image display function, it is highly effective to apply the information terminal device according to the present invention, and the mobile phone terminal has high added value. A terminal can be provided.

The sound data such as music includes various sounds that are audible, such as human voices such as conversations and lectures, singing sounds of insects and animals, in addition to music. Therefore, it is possible to associate image data or the like that forms an image of the insect's appearance with voice data of the insect's cry, and use both of them simultaneously.

In the above-described embodiment, the association information between audio data and image data is described as using the file name and extension of audio data and the file name and extension of image data. However, the present invention is not limited to this. For example, for audio data, ISRC
(International StandardRe
For example, unique identification information added to each piece of music (song), such as coding code, can be used. Therefore, as the association information, various types of information capable of reliably specifying each of the audio data and the image data can be used.

As the image data associated with audio data such as music, in addition to a still image, image data for forming a moving image, text data for forming characters, symbols, figures, and the like can be used.

Also, in the above-described embodiment, the case where audio data such as music and image data correspond to one body has been described as an example, but a group of audio data such as music (for example, Of course, a plurality of image data can be associated with one piece of audio data). Alternatively, a plurality of image data may be handled as one image data in a hierarchical structure, and may be associated with audio data such as music.

As described above, when a plurality of image data are associated with audio data such as music, the image to be displayed is changed every predetermined time, or an instruction from the user is issued. The display of a plurality of images associated with each other may be sequentially switched. In this case, for example, the user himself / herself can display and view a plurality of images associated with the audio data during the reproduction of one piece of music.

The image data includes a still image such as an idle photograph or an image photographed by using a camera function, a moving image, character information such as lyrics, symbols and figures, a still image, a moving image, and the like. One that forms various images, such as a combination of character information, symbols, and figures, can be used.

[0159]

As described above, according to the present invention, when sound data such as music is reproduced, it is possible to provide image information in accordance with the needs of the user. A form can be realized.
Further, when reproducing audio data such as music, the image display function of the mobile phone terminal can be fully utilized.

During reproduction of audio such as music, not only very limited information such as song titles but also an image selected by the user himself can be displayed, so that the display elements provided in the information terminal device can be effectively utilized. And you can spend more time playing music.

Further, it is possible to provide a new creative work of searching for an image that matches the user's favorite music, or photographing using a digital camera function, and associating them with each other. The added value of the telephone terminal can be increased.

[Brief description of the drawings]

FIG. 1 is a block diagram for explaining a mobile phone terminal to which an embodiment of an information terminal device according to the present invention is applied.

FIG. 2 is a view for explaining an appearance of the mobile phone terminal shown in FIG. 1;

FIG. 3 is a diagram for explaining an example of image information displayed on a display screen of the LCD shown in FIG.

FIG. 4 is a diagram for explaining association information recorded in a flash memory of the mobile phone terminal shown in FIG. 1;

FIG. 5 is a flowchart illustrating a reproduction process of audio data and image data and a process of associating audio data with image data, which are executed in the mobile phone terminal illustrated in FIG. 1;

6 is a diagram for explaining another example of the association information recorded in the flash memory of the mobile phone terminal shown in FIG.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Transmission / reception antenna, 2 ... Antenna sharing device, 3 ... Receiving part, 4-baseband processing part, 5 ... Voice codec (DSP), 6 ... Speaker (receiver), 7 ... Microphone (transmitter), 8 ... Transmission part, 9 ... frequency synthesizer,
11 audio data reproduction processing unit 12 audio signal output terminal 13 LCD controller 14 LCD 15
Camera unit, 16 linger, 20 control unit, 21 CP
U, 22 ROM, 23 SRAM, 24 flash memory, 25 CPU bus, 31 external memory interface, 32 external memory slot 32, 33 key interface, 34 key operation unit, 35 external interface, 36 ... I / O terminals, 40 ... External memory,
50 ... headphones

Claims (10)

[Claims] 1. An audio data reproducing means for forming a reproduced audio signal from audio data such as music, an output end of the reproduced audio signal from the audio data reproducing means, and image data for forming a reproduced image signal from image data. A reproducing unit, a display element that receives the reproduction image signal from the image data reproducing unit and displays an image corresponding to the reproduction image signal, and an instruction input for associating the audio data with the image data Accepting means for accepting, storing means for storing association information for associating the audio data and the image data according to the instruction input received through the accepting means, and when reproducing the audio data, Obtaining the image data associated with the audio data based on the association information stored in a storage unit; This was supplied to the image data reproducing means, the information terminal device characterized by comprising a display control unit to display the image by the image data to the display device. 2. The information terminal device according to claim 1, further comprising a built-in storage unit for storing the image data, wherein the display control unit obtains the image data from the built-in storage unit. Information terminal device. 3. The information terminal device according to claim 1, further comprising a connection end of a detachable storage unit that stores and holds the image data, wherein the display control unit is connected to the connection end. An information terminal device, wherein the image data is obtained from a possible storage unit. 4. The information terminal device according to claim 2, further comprising a communication unit for performing communication, wherein the image data is obtained through the communication unit, and the image data is stored and held. An information terminal device characterized in that: 5. The information terminal device according to claim 1, further comprising communication means for performing communication, wherein said display control means obtains said image data through said communication means. apparatus. 6. The information terminal device according to claim 4, wherein the communication unit connects to a wireless network and obtains the image data from a server device on the wireless communication network. Information terminal device. 7. The information terminal device according to claim 4, wherein the communication unit is connected to a wired network and obtains the image data from a server device on the wireless communication network. Information terminal device. 8. The information terminal device according to claim 4, wherein the communication unit obtains the image data from a device that holds the image data by a short-range wireless communication method. An information terminal device characterized by the above-mentioned. 9. The information terminal device according to claim 4, wherein the communication unit obtains the image data from a device that holds the image data through a wired digital interface. Characteristic information terminal device. 10. The information terminal device according to claim 1, wherein information indicating a storage location of the image data is added to the association information stored in the storage unit. The information terminal device obtains the target image data from a storage location indicated by information indicating a storage location added to the association information.