JP2008270914A - Control device, mobile communication system, and communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To promote communication between users in the presence of amusement and safety. <P>SOLUTION: Speech messages are sequentially acquired after a video telephone is started between two communication terminals, and each of the communication terminals is caused to display photographed images or avatar images as alternatives for the photographed images according to whether there is a word among a plurality of words registered beforehand, matching with or similar to a word included an acquired message. For example, the users each register a plurality of favorite words matching with his or her taste and idea in advance, start a video telephone using the alternative image first, then change the alternative images to the photographed images when one of the users utters a favorite word registered beforehand. It is considered that each user wants to see the face of a partner having the same interest, so that such user's feelings are guessed and the alternative images are switched to the photographed images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for performing communication between users using a message or an image by text or voice.

As mobile phones become more sophisticated in recent years, communication that is not based only on voice is possible. For example, a mobile phone having a so-called videophone function that transmits and receives an image of a face or the like taken with an attached camera together with sound has become widespread. In addition, a mobile phone that displays an image of a predetermined character or the like during a voice call is also known (see, for example, Patent Documents 1 and 2). By using such a technique, it becomes possible to perform communication that is more intimate and more entertaining than when only voice is used.
JP-T-2004-537231 JP 2004-297350 A

  By the way, when communicating with a party whose identity is not very clear using the videophone function, your face may be known without any restrictions. May not be protected. On the other hand, as shown in Patent Documents 1 and 2, simply continuing to display images of characters and the like during a call does not increase the feeling of intimacy between speakers, and there is a concern that communication is not promoted much.

  The present invention has been made in view of such circumstances, and an object of the present invention is to promote communication between speakers while restricting use of an image obtained by photographing the speakers.

  In order to achieve the above-described object, the present invention is transmitted and received between a storage means for storing a plurality of words and a first communication terminal and a second communication terminal that transmit and receive images and messages and display images. Determining whether there is a word that matches or is similar to a word included in the message acquired by the acquisition unit among the plurality of words stored in the storage unit and the acquisition unit that acquires the message And a substitute image in place of the captured image transmitted from the other communication terminal to at least one of the first communication terminal and the second communication terminal. There is provided a control device comprising control means for displaying any one of them in accordance with a determination result of the determination means.

In a preferred aspect of the present invention, when the transmission of the image and the message is started between the first communication terminal and the second communication terminal, the control means is configured to transmit the first communication terminal or the second communication terminal. After displaying the substitute image on at least one of the communication terminals and displaying the substitute image on the one communication terminal, among the plurality of words stored by the storage unit, When it is determined that there is a word that matches or is similar to the word included in the message acquired by the acquisition unit, the captured image is displayed on the one communication terminal.
In this aspect, it comprises history storage means for storing a history of transmission / reception of the image or the message made by the first communication terminal and the second communication terminal, and the control means transmits / receives data to / from the history storage means. When transmission / reception of the image and the message is started between the first communication terminal and the second communication terminal in which no history is stored, the first communication terminal or the second communication terminal The substitute image is displayed on at least one of the communication terminals, and the image and the second communication terminal between the first communication terminal and the second communication terminal in which the history of transmission / reception is stored in the history storage unit When transmission / reception of the message is started, the captured image is displayed on at least one of the first communication terminal and the second communication terminal. It may be allowed to.

  Moreover, in another preferable aspect of the present invention, when the transmission and reception of the image and the message are started between the first communication terminal and the second communication terminal, the control means is configured to transmit the first communication terminal. Alternatively, after the captured image is displayed on at least one of the second communication terminals and the captured image is displayed on the one communication terminal, a plurality of data stored by the storage unit is stored. When it is determined that there is a word that matches or is similar to the word included in the message acquired by the acquisition unit, the substitute image is displayed on the one communication terminal.

  Moreover, in another preferable aspect of the present invention, the first communication terminal and the second communication terminal are each the substitute image displayed by the communication terminal of the other party that is the transmission destination of the image or message from the own terminal. And when the control means displays the photographed image on at least one of the first communication terminal and the second communication terminal, the other communication terminal The other communication terminal is instructed to transmit the captured image obtained by shooting and the message, and the alternative communication terminal is replaced with at least one of the first communication terminal and the second communication terminal. When displaying an image, the other communication terminal is instructed to transmit the substitute image and the message stored in the other communication terminal.

  Moreover, in another preferable aspect of the present invention, each of the first communication terminal and the second communication terminal stores a plurality of substitute images that can be displayed as substitutes for the captured image of the other party in the storage unit of the own terminal. And when the control means displays the photographed image on at least one of the first communication terminal and the second communication terminal, the other communication terminal obtained by photographing. The other communication terminal is instructed to transmit the captured image and the message, and the substitute image is displayed on at least one of the first communication terminal and the second communication terminal. In this case, the alternative image designated by the other communication terminal is notified to the one communication terminal, and the notification is made of the plurality of alternative images stored in the storage unit of the one communication terminal. Instructs said one communication terminal to display an alternative image.

  In another preferable aspect of the present invention, the image processing apparatus further includes a substitute image storage unit that stores the substitute image, and the control unit is at least one communication terminal of the first communication terminal or the second communication terminal. When displaying the captured image on the other communication terminal, the other communication terminal instructs the other communication terminal to transmit the captured image obtained by shooting and the message, and the first communication terminal or the When the substitute image is displayed on at least one communication terminal of the second communication terminals, the substitute image is read from the substitute image storage unit and transmitted to the one communication terminal.

  In another preferable aspect of the present invention, the substitute image storage means stores an identifier assigned to the first communication terminal or the second communication terminal and the substitute image in association with each other. The control means associates an identifier assigned to the other communication terminal when displaying the substitute image on at least one of the first communication terminal and the second communication terminal. The stored alternative image is read from the alternative image storage means and transmitted to the one communication terminal.

  According to another aspect of the present invention, there is provided a storage device that stores a plurality of words, and the message that is transmitted and received between the first communication terminal and the second communication terminal that transmit and receive images and messages and display images. An acquisition device; a determination device that determines whether there is a word that matches or is similar to a word included in a message acquired by the acquisition device among a plurality of words stored in the storage device; Either one of the communication image of the first communication terminal or the second communication terminal, a captured image transmitted from the other communication terminal, or an alternative image replacing the captured image, There is provided a mobile communication system comprising: a control device that displays according to a determination result of the determination device.

  In another preferred embodiment of the present invention, an image is acquired between an acquisition unit that acquires a captured image captured by the capturing unit, a storage unit that stores an alternative image instead of the captured image, and a communication terminal that is a communication destination. Transmission / reception means for transmitting / receiving messages, display means for displaying images received by the transmission / reception means, storage means for storing a plurality of words, and transmission / reception among a plurality of words stored by the storage means Determining means for determining whether or not there is a word that matches or is similar to a word included in the message received by the means, and the captured image acquired by the acquiring means or the storage according to the determination result of the determining means Selecting means for selecting any one of the substitute images stored in the means as an image to be transmitted by the transmission / reception means. To provide a terminal.

  In addition, the present invention provides an acquisition unit that acquires a captured image captured by the imaging unit, a transmission / reception unit that transmits / receives an image or data and a message, and a reception / reception unit that receives the image or data. A display means for displaying the received image, a storage means for storing a plurality of words, and a word included in the message received by the transmission / reception means among the plurality of words stored in the storage means. A determination unit that determines whether or not there is a word to be used, and designation data that specifies a captured image acquired by the acquisition unit or a substitute image that is a substitute for the captured image, according to a determination result of the determination unit And a selecting means for selecting any of the above as an image or data transmitted by the transmitting / receiving means.

  In a preferred aspect of the present invention, the image processing apparatus includes a storage unit that stores a plurality of the substitute images, and the display unit includes a substitute image stored in the storage unit, instead of the image received by the transmission / reception unit. The substitute image designated by the designated data received by the transmission / reception means is displayed.

  According to the present invention, an image obtained by photographing a speaker of a communication terminal by switching and displaying a captured image transmitted from the other communication terminal or an alternative image that is a substitute for the captured image on one communication terminal. While restricting the use of, communication between speakers can be promoted.

Next, embodiments of the present invention will be described with reference to the drawings.
In the following description, a voice call with an image is referred to as “videophone”.
The term “image” includes both the meanings of still images and moving images.
The “photographed image” is an image obtained by photographing, and is typically a user's face image photographed by a camera attached to the communication terminal.
“Avatar” is a symbolic representation of a user of a communication terminal. This avatar is relatively often used in interactive communication services such as chat and bulletin boards on the Internet. The “avatar image” is an image representing an avatar, and includes, for example, an image imitating a human face or figure, an image of an animal, a robot, or a virtual life form.

In the present embodiment, voice messages from when a videophone call is started between two communication terminals are acquired one by one, and a word included in the acquired message is matched with a plurality of preregistered words or Depending on whether or not there is a similar word, either the captured image or an avatar image serving as a substitute for the captured image is displayed on each communication terminal.
For example, a plurality of words preferred for one's own hobbies are registered in advance, first a videophone call is started using a substitute image, and then a preferred word registered in advance by one of the other parties When it is emitted, the substitute image is switched to the captured image. For the user, it seems that there is a need to look at the other person's face as a person who meets his / her hobby, so it is recommended to switch from the substitute image to the captured image based on the user's feelings. Is.
In addition, for example, a plurality of inappropriate or unfaithful words are registered in advance in light of common sense, such as words that the other party feels uncomfortable or discrimination terms. First, a videophone call is started using each other's photographed image. After that, if an inappropriate word registered in advance is issued from either one of the other parties, the photographed image is switched to the substitute image. The user does not want to talk while looking at the face of the other person who feels uncomfortable, so that the user's feeling is inferred and the captured image is switched to the substitute image.

[Constitution]
FIG. 1 is a diagram schematically showing an overall configuration of a system according to an embodiment of the present invention. As shown in the figure, this system includes mobile communication terminals 10A and 10B, a mobile communication network 20, and a control device 30. Although many mobile communication terminals can exist in this system, FIG. 1 shows only the mobile communication terminals of the communication source and the communication destination. In the following, the mobile communication terminal 10A is the source mobile communication terminal, and the mobile communication terminal 10B is the destination mobile communication terminal. It is referred to as “communication terminal 10”.

  The mobile communication network 20 is a network system for providing a mobile communication service to the mobile communication terminal 10, and is managed by a communication carrier called a carrier. The mobile communication network 20 includes nodes such as a base station, an exchange station, and a service control station (not shown), and a communication line that connects these nodes. Voice data, image data, Control data is multiplexed and transmitted. For example, 3G-324M standardized by 3GPP (3rd Generation Partnership Project) is suitable as a protocol that the mobile communication network 20 complies with. The service control station stores location registration information of the mobile communication terminal 10, information related to a contract and billing for the user of the mobile communication terminal 10, a telephone number of each mobile communication terminal 10, and the like. A control device 30 is connected to the service control station of the mobile communication network 20.

  FIG. 2 is a block diagram illustrating a configuration of the control device 30. As shown in the figure, the control device 30 includes a control unit 31, a storage unit 33, and a communication unit 32. The control unit 31 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU executes a program stored in the ROM or the storage unit 33 using the RAM as a work area. Thus, the operation of each part of the control device 30 is controlled. The storage unit 33 includes a storage device such as an HDD (Hard Disk Drive). The storage unit 33 stores various data for realizing communication between the mobile communication terminals 10 in addition to the program executed by the control unit 31. The communication unit 32 is an interface device for performing communication via the mobile communication network 20.

Here, data stored in the storage unit 33 will be described.
The storage unit 33 stores various data for performing processing to be described later, such as a map file, position data of a plurality of avatars, an avatar ID conversion table, a call history, a term database, and avatar image data. I remember it. The map file is a set of data representing a virtual three-dimensional space (hereinafter referred to as “virtual space”), and includes object data, position information, and path information. The object data is data representing objects such as buildings and roads that make up the virtual space. The object data is data that defines the shape and color of each object, that is, the appearance, and is so-called polygon data. This object is only a static object, that is, an object whose position in the virtual space is fixed, and does not include a dynamic object such as an avatar. The position information represents a position in the virtual space based on a two-dimensional or three-dimensional coordinate system. Each object represented by the object data is associated with position information. The path information is data that defines a place that can be an avatar path in the virtual space. The place where the path information is defined is, for example, a road.

  The avatar position data is data in which avatar position data transmitted from the plurality of mobile communication terminals 10 to the control device 30 is stored. As shown in FIG. 3, the avatar ID conversion table is a table in which an avatar ID assigned to each avatar is associated with a telephone number of a user's mobile communication terminal symbolized by the avatar. . The call history is a history of calls between the mobile communication terminals 10 and includes a telephone number of a communication source mobile communication terminal, a communication destination mobile communication terminal, a time when the call was made, and the like. The term database includes terms preferable for the user of the mobile communication terminal 10. This term is notified in advance by the user of the mobile communication terminal 10, and is stored in association with the telephone number of each mobile communication terminal 10 as shown in FIG.

Next, the mobile communication terminal 10 will be described.
The mobile communication terminal 10 is a so-called mobile phone, which is a communication terminal capable of performing a call and communication with another mobile communication terminal 10 via the mobile communication network 20. In addition, the mobile communication terminal 10 has a videophone function, and is configured to transmit and receive images while performing a voice call and to display received images. Furthermore, the mobile communication terminal 10 is configured to display a virtual space realized by the control device 30 and to perform communication via other avatars in the virtual space by moving an avatar representing the user. Yes.

FIG. 5 is a block diagram showing a configuration of the mobile communication terminal 10.
As shown in the figure, the mobile communication terminal 10 includes a control unit 11, a radio communication unit 12, an operation unit 13, a display unit 14, a voice input / output unit 15, a photographing unit 16, and a multimedia processing unit. 17. The control unit 11 includes a CPU 11a, a ROM 11b, a RAM 11c, and an EEPROM (Electronically Erasable and Programmable ROM) 11d. The CPU 11a uses the RAM 11c as a work area to execute a program stored in the ROM 11b or the EEPROM 11d, thereby the mobile communication terminal 10 Control the operation of each part. The wireless communication unit 12 includes an antenna 12a and wirelessly transmits and receives data to and from the mobile communication network 20. The operation unit 13 includes an operation element such as a button, and supplies an operation signal corresponding to a user operation to the control unit 11. The display unit 14 is a display device that includes a liquid crystal panel and a liquid crystal driving circuit, and displays various types of information according to instructions from the control unit 11. The voice input / output unit 15 includes a microphone 15a and a speaker 15b, and inputs and outputs a voice signal. The photographing unit 16 has a so-called camera function. The photographing unit 16 includes a complementary metal oxide semiconductor (CMOS) image sensor and a signal processing circuit, and generates image data representing a photographed subject.

  The multimedia processing unit 17 includes an LSI (Large Scale Integration) for processing data transmitted and received via the wireless communication unit 12, and includes audio and image encoding / decoding processing, multiplexing processing, separation processing, and the like. Execute. The multimedia processing unit 17 generates moving image data based on the image data generated by the photographing unit 16. This moving image data is hereinafter referred to as “photographed image data”. An AMR (Adaptive Multi-Rate) may be used for encoding / decoding an audio signal, and a Moving Picture Experts Group (MPEG) -4 may be used for encoding / decoding image data.

  Here, the controls provided in the operation unit 13 will be described with reference to FIG. As shown in the figure, the operation unit 13 includes a function button Bs, movement buttons Bu, Bd, Bl, Br, a confirmation button Bf, and dial buttons B1 to B0. The function button Bs is a button to which a predetermined function corresponding to the screen display of the display unit 14 is assigned. The function assigned to the function button Bs is a function for selecting a communication destination, and details thereof will be described later. The movement buttons Bu, Bd, B1, and Br are buttons for moving the object to be moved (avatar or pointer) forward, backward, left, right (or up, down, left, right). The confirmation button Bf is a button for selecting an object displayed on the display unit 14 and confirming the content of the subsequent processing. The dial buttons B1 to B0 are buttons for inputting characters and numbers.

  Then, the content of the data which the mobile communication terminal 10 memorize | stores is demonstrated. The ROM 11b stores some programs in advance. Hereinafter, this is referred to as a “preinstalled program”. Specifically, the preinstalled programs are programs of a multitask operating system (hereinafter referred to as “multitask OS”), a Java (registered trademark) platform, and a native application. To outline these programs, first, the multitasking OS is an operating system that supports various functions such as allocation of virtual memory space necessary to realize pseudo-parallel execution of multiple tasks by TSS (Time-Sharing System). System. The Java platform is a program group described in accordance with a CDC (Connected Device Configuration) that is a configuration for realizing a Java execution environment 114 described later in a portable device equipped with a multitask OS. The native application is a program that realizes basic services of the mobile communication terminal 10 such as calling, communication, and camera shooting.

  The EEPROM 11d has a Java application storage area in which Java applications are stored. A Java application is a JAR (Java Archive) file that combines an entity program that describes the processing procedure itself under the Java execution environment 114 and an image file and an audio file that are used in accordance with the execution of the entity program. It has an ADF (Application Descriptor File) that describes installation and activation of a JAR file and various attributes. The Java application is created by a content provider or a communication provider, stored in a server device connected to the Internet or the mobile communication network 20, and downloaded from the control device as needed according to a request from the mobile communication terminal 10. It has come to be.

  FIG. 7 is a diagram illustrating a logical configuration of each unit implemented in the control unit 11 of the mobile communication terminal 10 by executing various programs in the ROM 11b and the EEPROM 11d. As shown in the figure, in the mobile communication terminal 10 that executes various programs, a communication application 112, a camera photographing application 113, and a Java execution environment 114 are realized on the OS 111, and the EEPROM 11d includes the first storage 115 and the first storage 115. 2 storage 116 is secured. The communication application 112 and the camera photographing application 113 are realized by a native application in the ROM 11b, and realize functions such as establishment of communication with the mobile communication network 20 and photographing of an image by the photographing unit 16, respectively.

  The Java execution environment 114 is realized by the Java platform in the ROM 11b. The Java execution environment 114 includes a class library 117, a JVM (Java Virtual Machine) 118, and a JAM (Java Application Manager) 119. The class library 117 is a group of program modules (classes) having specific functions combined into one file. The JVM 118 is a Java execution environment optimized for the above-described CDC, and has a function of interpreting and executing bytecode provided as a Java application. The JAM 119 has a function of managing download, installation, activation / termination, and the like of Java applications. The first storage 115 is an area for storing Java applications (Jar files and ADFs) downloaded under the management of the JAM 119. The second storage 116 is an area for storing data generated when the Java application is executed after the end thereof, and a separate storage area is assigned to each installed Java application. . The data in the storage area allocated to a certain Java application can be rewritten only while the Java application is being executed, and another Java application cannot be rewritten.

The Java application includes an application for displaying a virtual space according to the movement of the avatar and performing a call and communication with another mobile communication terminal 10 according to a user instruction. This application is hereinafter referred to as a “videophone application”.
This videophone application is stored in advance in the mobile communication terminal 50. Further, the EEPROM 11d includes “avatar image data” representing an avatar necessary for executing the videophone application.

  When making a videophone call using the avatar image data, the control unit 11 reads the avatar image data from the EEPROM 11 d and transmits it to the mobile communication network 20 together with the voice message input to the voice input / output unit 15. The mobile communication network 20 transmits the avatar image data and the voice message to the mobile communication terminal 10 that is the communication destination. In the mobile communication terminal 10 that is the communication destination, the control unit 11 displays the avatar image on the display unit 14 based on the avatar image data received by the wireless communication unit 12, and the received voice message is sent to the voice input / output unit 15. Output from. FIG. 8A is a diagram illustrating an example in which the avatar image represented by the avatar image data is displayed on the display unit 14 of the mobile communication terminal 10 that is the communication destination.

  On the other hand, when making a videophone call using the captured image data, the control unit 11 uses the mobile communication network to capture the captured image data obtained by capturing by the capturing unit 16 together with the voice message input to the voice input / output unit 15. 20 to send. The mobile communication network 20 transmits the captured image data and the voice message to the mobile communication terminal 10 that is the communication destination. In the mobile communication terminal 10 that is the communication destination, the control unit 11 displays the captured image on the display unit 14 based on the captured image data received by the wireless communication unit 12 and the received voice message is transmitted to the voice input / output unit 15. Output from. FIG. 8B is a diagram illustrating an example in which the face of the user of the mobile communication terminal 10 photographed by the photographing unit 16 is displayed on the display unit 14 of the mobile communication terminal 10 that is the communication destination.

[Operation]
Next, operations of the mobile communication terminal 10 and the control device 30 in the mobile communication system having the above configuration will be described. Here, the process executed when the mobile communication terminal 10 starts the above-described videophone application will be described first, and then the mobile communication terminal 10 performs a voice call between the mobile communication terminal 10A and the mobile communication terminal 10B. Processing executed by each of 10A, 10B and the control device 30 will be described.

  FIG. 9 is a flowchart showing processing executed when the mobile communication terminal 10A activates the videophone application. When the videophone application is activated in response to a predetermined operation by the user, the control unit 11 of the mobile communication terminal 10A controls data indicating a predetermined position in the virtual space and data indicating the telephone number of the terminal itself. It transmits to the apparatus 30 (step Sa1). The position indicated by the position data is arbitrary. For example, the position data may be a predetermined position, or may be the position of an avatar when the execution of the videophone application has been completed previously.

  In the control device 30 that has received the position data, the control unit 31 acquires this, and also refers to the map file stored in the storage unit 33, and specifies object data corresponding to the acquired position data. Specifically, the control unit 31 specifies object data within a predetermined range from the position indicated by the acquired position data. When the object data is specified in this way, the control unit 31 transmits the object data and the position data of each object data to the mobile communication terminal 10A. At this time, when an avatar of another user exists within the predetermined range described above, the control unit 31 displays the avatar image data representing the avatar, the avatar ID assigned to the avatar, and the avatar. Together with position data indicating the position of the avatar is transmitted as avatar-related information. The control unit 11 of the mobile communication terminal 10A acquires object data and avatar related information from the control device 30 (step Sa2), and displays an image representing the virtual space on the display unit 14 based on the object data and the avatar related information (step Sa3).

  FIG. 10 is a diagram illustrating an image displayed on the display unit 14 at this time. In the figure, an image D0 is an avatar image representing the user of the mobile communication terminal 10A, and moves in the virtual space according to the operation of the user of the mobile communication terminal 10A. The images D1, D2, and D3 represent buildings, and a region P1 sandwiched between the images D1, D2, and D3 represents a road. The image D4 is an avatar image representing the user of the mobile communication terminal 10B other than the mobile communication terminal 10A, and moves in the virtual space according to the operation of the user of the mobile communication terminal 10B. The image D5 is an image representing the function assigned to the function button Bs.

  When the user presses the movement buttons Bu, Bd, Bl, and Br in this state, the control unit 11 changes the display on the display unit 14 so that the position of the avatar in the virtual space changes. For example, when the user presses the movement button Bu in the state shown in FIG. 10, the control unit 11 displays on the screen in which the display positions of the images D1, D2, D3, and D4 displayed based on the object data are changed. By updating the display content of the unit 14, the avatar of the image D0 is shown as if it is moving forward. When the user presses the function button Bs in this state, the control unit 11 displays a pointer on the display unit 14 and selects an avatar corresponding to the communication destination. When the user presses the function button Bs while the pointer is displayed, the control unit 11 hides the pointer so that the avatar can be moved again.

  FIG. 11 is a diagram illustrating an image when the display unit 14 displays a pointer. In the figure, an arrow-shaped image D6 represents a pointer. As described above, when the user presses the movement buttons Bu, Bd, B1, and Br while the pointer is displayed, the control unit 11 changes the display of the display unit 14 so that the position of the pointer changes. That is, the move buttons Bu, Bd, Bl, Br function as an operator for moving the avatar when the pointer is not displayed, and function as an operator for moving the pointer when the pointer is displayed. . When the confirmation button Bf is pressed in a state where the avatar image corresponding to the desired communication destination and the pointer overlap each other, the control unit 11 requests to make a videophone call with the communication destination corresponding to the avatar. Is transmitted to the control device 30.

Here, the above-described operation will be described with reference to the flowchart of FIG. If the image display of step Sa3 is performed, the control part 11 will judge whether the movement of the avatar is instruct | indicated by the user (step Sa4). Specifically, the control unit 11 repeatedly determines whether or not an operation signal corresponding to any of the movement buttons Bu, Bd, B1, or Br is supplied. Then, when the movement of the avatar is instructed by the user (step Sa4: YES), the control unit 11 transmits position data indicating the position of the avatar to the control device 30 (step Sa5), other than the already acquired object data. It is determined whether or not new object data is necessary (step Sa6). If new object data is unnecessary (step Sa6: No), the control unit 11 updates the display position of the image displayed based on the object data so that the avatar moves (step Sa7). ), The process returns to step Sa4 again. If new object data is necessary (step Sa6: Yes), the control unit 11 acquires object data corresponding to the position data from the control device 30 again (step Sa2).
The control unit 11 repeats the above process while the user moves the avatar.

  On the other hand, when the movement of the avatar is not instructed by the user (step Sa4: NO), the control unit 11 determines whether the communication destination is selected by the user (step Sa8). Specifically, the control unit 11 determines whether an operation signal corresponding to the confirmation button Bf is supplied in a state where the avatar image and the pointer overlap each other. If the communication destination is not selected by the user (step Sa8: NO), the control unit 11 returns to the process of step Sa4 again. On the other hand, when the communication destination is selected (step Sa8: YES), the control unit 11 executes a process for making a videophone call (step Sa9). This processing (hereinafter referred to as “videophone processing”) will be described in detail later. Thereafter, the control unit 11 determines whether or not the videophone application has been terminated by the user (step Sa10). If the user instructs the termination (step Sa10: YES), the control unit 11 terminates the videophone application. If the end is not instructed (step Sa10: NO), an image representing the virtual space is displayed again on the display unit 14, and the processing after step Sa3 is repeated.

  Next, the videophone process in step Sa9 will be described. This process will be described together with the processes in the control device 30 and the mobile communication terminal 10B that is the communication destination. FIG. 12 is a sequence chart showing a series of processes executed by the mobile communication terminals 10A and 10B and the control device 30 at this time. Hereinafter, the operations of the mobile communication terminals 10A and 10B and the control device 30 will be described with reference to FIG.

  First, the control unit 11 of the mobile communication terminal 10A transmits a request for making a videophone call to the control device 30 (step Sb1). This request includes the telephone number of the mobile communication terminal 10A and the avatar ID of the avatar designated as the communication destination by the pointer D6. When acquiring the request via the communication unit 32, the control unit 31 of the control device 30 converts the avatar ID included in the request into a telephone number using the avatar ID conversion table stored in the storage unit 33 (step Sb2). ). Then, the control unit 31 searches the call history in the storage unit 33 using the combination of the telephone number of the mobile communication terminal 10A and the communication destination telephone number as a key, and whether there is a match with the set in the call history Whether or not is checked (step Sb3).

  Subsequently, the control unit 31 determines image data to be displayed on the mobile communication terminals 10A and 10B when performing a videophone call based on the search result in step Sb3 (step Sb4). Specifically, when the call history includes a set of the telephone number of the mobile communication terminal 10A and the telephone number of the communication destination, the control unit 31 performs the mobile communication terminal 10A and the mobile communication terminal 10B when making a videophone call. The image data to be displayed on the screen is determined to be the captured image data of each other. On the other hand, when the call history does not include the combination of the telephone number of the mobile communication terminal 10A and the telephone number of the communication destination, the control unit 31 displays the mobile communication terminal 10A and the mobile communication terminal 10B when performing a videophone call. The determination is made that the image data is mutual avatar image data.

  Next, based on the determination in step Sb4, the control unit 31 transmits data indicating image data to be transmitted to the mobile communication terminal 10A (step Sb5). That is, when the call history does not include the set of the telephone number of the mobile communication terminal 10A and the telephone number of the communication destination, the control unit 31 instructs the mobile communication terminal 10A to transmit avatar image data. I do. In parallel with this, the control unit 31 instructs each node such as a service control station of the mobile communication network 20 to make a call to the mobile communication terminal 10B (step Sb6). This call includes a videophone call using avatar image data and avatar image data representing the user of the mobile communication terminal 10A.

  In the mobile communication terminal 10A that has received the above instruction from the control device 30, the control unit 11 displays a screen as shown in FIG. 13 on the display unit 14 (step Sb7). This screen indicates that the other party is being called to make a videophone call using the avatar image. On the other hand, in the called mobile communication terminal 10B, the control unit 11 displays a screen as shown in FIG. 14 on the display unit 14 (step Sb8). On this screen, an avatar image of the caller user and a message for inquiring whether or not to make a videophone call using the avatar image are described. At this time, if the user selects “YES”, the control unit 11 of the mobile communication terminal 10B returns a response to the mobile communication network 20, thereby establishing a communication line between the mobile communication terminal 10A and the mobile communication terminal 10B. (Step Sb9).

  After the call line is established, the control unit 11 of the mobile communication terminals 10A and 10B displays an image as shown in FIG. 8A on the display unit 14, and a videophone call is made in this state. . That is, the control unit 11 of each of the mobile communication terminals 10A and 10B reads the avatar image data from the EEPROM 11d and transmits it to the mobile communication network 20 together with the voice message input to the voice input / output unit 15. The mobile communication network 20 transmits the avatar image data and the voice message to the counterpart mobile communication terminal. In the other party's mobile communication terminal 10, the control unit 11 displays the avatar image on the display unit 14 based on the avatar image data received by the wireless communication unit 12, and receives the received voice message from the voice input / output unit 15. Output.

  Simultaneously with the establishment of the communication line, the control unit 31 of the control device 30 starts a voice recognition process for a voice message transmitted / received between the mobile communication terminal 10A and the mobile communication terminal 10B. Then, the words recognized by the speech recognition process are compared with the words included in the term database, and there is a word that matches or is similar to the words recognized by the speech recognition process among the words included in the term database. It is determined whether or not (step Sb10). The “similar” range here is a range that is defined in advance in consideration of the accuracy of speech recognition processing, the deformation of terms, etc., and is a range that is considered to be a synonym for a word included in the term database. is there.

At this time, words included in the voice message transmitted from the mobile communication terminal 10A are compared with words stored in the term database in association with the telephone number of the mobile communication terminal 10B. Further, the words included in the voice message transmitted from the mobile communication terminal 10B are compared with the words stored in the term database in association with the telephone number of the mobile communication terminal 10A. In other words, it is judged whether or not the word uttered by the opponent matches his / her preference.
When the user selects “NO” on the screen displayed in step Sb8, the control unit 11 of the mobile communication terminal 10B returns a response to the mobile communication network 20 to reject the incoming call. At this time, the communication line between the mobile communication terminal 10A and the mobile communication terminal 10B is not established.

When a certain number or more of words included in the term database that match or are similar to the words recognized by the speech recognition process are detected (step Sb11), the control unit 31 uses a captured image. The fact that the videophone is switched to the videophone using the photographed image is transmitted to the mobile communication terminal 10A and the mobile communication terminal 10B (steps Sb12 and Sb13).
“At least a certain number” at this time matches the word contained in the voice message transmitted from the mobile communication terminal 10A among the words stored in association with the telephone number of the mobile communication terminal 10B. Or, the number of similar words is equal to or greater than a certain number and is included in the voice message transmitted from the mobile communication terminal 10B among the words stored in association with the telephone number of the mobile communication terminal 10A. It means that the number of words that match or are similar to a word is a certain number or more.

  In the mobile communication terminal 10A that has received the above instruction from the control device 30, the control unit 11 displays a screen as shown in FIG. 15 on the display unit 14 (step Sb14). On this screen, a message for inquiring whether to make a videophone call using a photographed image is written. If the user selects “YES” at this time, the control unit 11 of the mobile communication terminal 10A returns a response to that effect to the mobile communication network 20 (step Sb15).

  On the other hand, the control unit 11 of the mobile communication terminal 10B also displays a screen similar to the above (step Sb16), and when the user selects “OK”, the control unit 11 returns a response to that effect to the mobile communication network 20. (Step Sb17). When the control unit 31 receives a response indicating OK from both mobile communication terminals 10A and 10B, the mobile communication terminal 10A and the mobile communication terminal 10B switch the video phone using the avatar image to the video phone using the photographed image. (Steps Sb18 and Sb19). In response to this instruction, the mobile communication terminal 10A and the mobile communication terminal 10B perform a videophone call using the photographed image as shown in FIG. 8B through the processing described above (step Sb20). That is, the control units 11 of the mobile communication terminal 10 </ b> A and the mobile communication terminal 10 </ b> B each transfer the captured image data obtained by the imaging by the imaging unit 16 to the mobile communication network 20 together with the voice message input to the voice input / output unit 15. Send. The mobile communication network 20 transmits the captured image data and the voice message to the mobile communication terminal 10 of the other party. In the counterpart mobile communication terminal 10, the control unit 11 displays a captured image on the display unit 14 based on the captured image data received by the wireless communication unit 12, and receives the received voice message from the voice input / output unit 15. Output.

  When the user selects “NO” on the screens displayed in steps Sb14 and Sb16, the control unit 11 of the mobile communication terminal 10 returns an incoming call rejection response to the mobile communication network 20. The mobile communication network 20 notifies the other party mobile communication terminal 10 of this fact, and in response to the notification, the control unit 11 of the mobile communication terminal 10 displays a screen as shown in FIG. At this time, the communication line between the mobile communication terminal 10A and the mobile communication terminal 10B is not established.

  In step Sb3, if the call history includes the set of the telephone number of the mobile communication terminal 10A and the telephone number of the communication destination, the control unit 31 transmits the captured image data to the mobile communication terminal 10A. Instruct to do so. In parallel with this, the control unit 31 instructs each node such as a service control station of the mobile communication network 20 to make a call to the mobile communication terminal 10B. This call includes a videophone call using photographed image data. In the mobile communication terminal 10A that has received the above instruction from the control device 30, a screen on which the control unit 11 is calling the other party to make a videophone call using the captured image (see FIG. 17). Is displayed (step Sb7).

On the other hand, in the called mobile communication terminal 10B, the control unit 11 describes the telephone number of the calling mobile communication terminal and a message asking whether or not the video call can be performed using the photographed image. Is displayed on the display unit 14 (step Sb8). When the user selects “OK” at this time, the control unit 11 of the mobile communication terminal 10B returns a response to the mobile communication network 20, thereby establishing a communication line between the mobile communication terminal 10A and the mobile communication terminal 10B. Is done.
Thereafter, a call is made between the mobile communication terminal 10A and the mobile communication terminal 10B with the captured image displayed.

  The operations of the mobile communication terminals 10A and 10B and the control device 30 in the present embodiment are as described above. By operating in this way, the user of the mobile communication terminal 10 can freely move in the virtual space via the avatar and make a videophone call with the user of the mobile communication terminal 10 who operates other avatars. . That is, according to the present embodiment, it is possible to make a videophone call without knowing the telephone number of the communication destination. Moreover, since only the avatar ID corresponding to the user who is the other party of the call is provided to the mobile communication terminal 10 and the telephone number of the other party is not provided, the situation where the telephone number is leaked is prevented. It is also possible. In addition, according to this embodiment, since a call is made by a method completely different from a conventional videophone, such as “searching for a conversation partner while walking around the virtual space in a pseudo manner using an avatar”, the videophone is highly entertaining. It can be expected that it will be imparted and promoted.

  In the above embodiment, the avatar image of the other party is displayed at the start of the call, and the avatar image is displayed on the captured image when the other party often utters words that suit his / her preference in the process of continuing the call. Switched and displayed. In this way, at first, a videophone call is displayed with an avatar image displayed, and when it can be predicted that the user will feel to some extent, a captured image is displayed to make a videophone call, thereby gradually changing the mode of communication. It is possible to change. In addition, by doing so, it can be expected that the user's psychological barrier to communication with a partner whose identity is not so clear will be eased. In addition, since the user is confirmed whether to switch from the avatar image to the captured image, the user of the mobile communication terminal 10 can also prevent the captured image from being displayed for an unintended partner. Thereby, a user's privacy can be protected appropriately.

[Modification]
The above embodiment may be modified as follows. Note that the following modifications may be combined as appropriate.
(1) Modification 1
In the above-described embodiment, a plurality of words preferred for one's own hobbies are registered in advance. First, a videophone call is started using a substitute image, and thereafter, one of the other parties registers in advance. When a favorable word is issued, the substitute image is switched to the photographed image.
Instead, for example, register a number of inappropriate or unfaithful words in advance against common sense, such as words that make the other person feel uncomfortable or discriminatory terms, and start a videophone call using each other's images. Thereafter, when an inappropriate word registered in advance is issued from either one of the partners, the captured image may be switched to the substitute image. In this case, the term database may be preliminarily registered with a plurality of inappropriate or unfaithful words, such as words that users feel uncomfortable or discriminatory terms, according to common sense. Broadcast prohibited terms may be used.
In the embodiment, the telephone number and the term of the mobile communication terminal 10 are stored in association with each other. However, since words and discrimination terms that are uncomfortable for the user are often common to the respective users, the mobile communication terminal 10 The telephone number and the term may not be stored in association with each other.
Further, in the embodiment, the telephone number and the term of the mobile communication terminal 10 are stored in association with each other so as to accurately match the user's hobbies and preferences. Only words that are favorable to the user may be stored in the term database.

(2) Modification 2
In the above-described embodiment, when the set of the telephone number of the communication source mobile communication terminal 10 and the communication destination mobile communication terminal 10 is included in the call history stored in the control device 30, The image to be displayed on the mobile communication terminal 10 when starting is taken as a captured image. However, it is not essential to consider the presence / absence of the call history when determining an image to be displayed when starting a videophone call.

(3) Modification 3
The above-described embodiment has a configuration in which an image to be displayed when performing a videophone call is switched on the mobile communication terminal 10 side of the transmission source (communication source), but such a configuration is not necessarily limited.
For example, both the avatar image data and the captured image data are transmitted to the transmission source mobile communication terminal 10, and one of these is selected on the control device 30 side according to the elapsed time, and the transmission destination (communication destination) is selected. It is good also as a structure which transmits to the mobile communication terminal 10. FIG.
The control device 30 may transmit both the avatar image data and the captured image data to the destination mobile communication terminal 10 and may instruct the destination mobile communication terminal 10 to display the image data to be displayed. In this case, the destination mobile communication terminal 10 may perform display using the image data instructed by the control device 30 among the received avatar image data and photographed image data.

(4) Modification 4
In the above-described embodiment, the mobile communication terminal 10 stores the avatar image data. However, the present invention is not limited to this.
For example, the control device 30 may store avatar image data in the storage unit 33 in advance. In this case, the control device 30 instructs the transmission source mobile communication terminal 10 to always transmit the captured image data and the message. Then, the control device 30 selects and transmits the captured image data received from the transmission source mobile communication terminal 10 or the avatar image data stored by itself as the image data to be transmitted to the transmission destination mobile communication terminal 10. To do. In order to realize this, the control device 30 needs to have a configuration corresponding to a multimedia processing unit of the mobile communication terminal 10 while relaying data (audio data, image data, etc.) related to a videophone. When the avatar image data is transmitted to the destination mobile communication terminal 10, the control unit 31 of the control device 30 acquires the audio data and the captured image data multiplexed via the communication unit 32. Are separated into individual audio data and photographed image data. Then, the control unit 31 of the control device 30 replaces the acquired captured image data with the avatar image data read from the storage unit 33, performs multiplexing processing, and then supplies the data to the communication unit 32 to send the destination mobile communication. Transmit to the terminal 10.

Alternatively, the control device 30 may store a plurality of avatar image data, and the mobile communication terminal 10 may be configured to transmit only specified data including an avatar ID for identifying the avatar image data. In this way, it is possible to reduce the amount of data transmitted in the source mobile communication terminal 10. In order to realize this, the control device 30 relays data related to the videophone and stores a plurality of avatar image data in association with the avatar ID in the storage unit 33. When an avatar image is displayed on the destination mobile communication terminal 10, the transmission source mobile communication terminal 10 transmits designated data including the avatar ID together with the voice message. When receiving the designated data from the transmission source mobile communication terminal 10, the control unit 31 of the control device 30 performs multiplexing processing on the avatar image data corresponding to the avatar ID included in the designated data, and then supplies the multiplexed data to the communication unit 32. Then, the data is transmitted to the destination mobile communication terminal 10. On the other hand, when the captured image is displayed on the mobile communication terminal 10 as the transmission destination, the control device 30 may instruct the transmission source mobile communication terminal 10 to transmit the captured image data and the message.
Furthermore, the control device 30 may store a plurality of avatar image data in association with the telephone number of each mobile communication terminal 10. When an avatar image is displayed on the destination mobile communication terminal 10, the source mobile communication terminal 10 transmits its own telephone number and voice message. The control unit 31 of the control device 30 performs a multiplexing process on the avatar image data corresponding to the telephone number of the transmission source mobile communication terminal 10, supplies the multiplexed data to the communication unit 32, and sends it to the transmission destination mobile communication terminal 10. To send. On the other hand, when the captured image is displayed on the mobile communication terminal 10 as the transmission destination, the control device 30 may instruct the transmission source mobile communication terminal 10 to transmit the captured image data and the message. Even in this case, it is possible to reduce the amount of data transmission in the mobile communication terminal 10 as the transmission source.

  The destination mobile communication terminal 10 may store a plurality of avatar image data that can be displayed as an avatar image symbolizing the user of the transmission source. This avatar image data is stored in association with each avatar ID. When an avatar image is displayed on the destination mobile communication terminal 10, the transmission source mobile communication terminal 10 transmits designated data including the avatar ID together with the voice message. The control device 30 transmits the designated data and the voice message to the mobile communication terminal 10 that is the transmission destination. The destination mobile communication terminal 10 reads out the avatar image data corresponding to the avatar ID included in the received designated data from the stored avatar images and displays the avatar image. On the other hand, when the captured image is displayed on the mobile communication terminal 10 as the transmission destination, the control device 30 may instruct the transmission source mobile communication terminal 10 to transmit the captured image data and the message.

(5) Modification 5
In the embodiment, the control device 30 performs comparison with the term database, but the mobile communication terminal 10 may perform this.
That is, a plurality of words are stored in the EEPROM 11d of the mobile communication terminal 10, and the word included in the voice message received from the counterpart mobile communication terminal 10 by the control unit 11 among the plurality of words stored in the EEPROM 11d. It is determined whether there is a word that matches or is similar to. Then, according to the determination result, either the photographed image or the avatar image received from the counterpart mobile communication terminal 10 is selected as the image transmitted from the wireless communication unit 12. The avatar image at this time may be stored by the mobile communication terminal 10 or may be stored by the control device 30.
When the avatar image is stored by the mobile communication terminal 10, the mobile communication terminal 10 communicates with an acquisition unit that acquires a captured image captured by the imaging unit, a storage unit that stores a substitute image instead of the captured image, and a communication unit. A transmission / reception unit that transmits and receives images and messages to and from a previous communication terminal, a display unit that displays images received by the transmission / reception unit, a storage unit that stores a plurality of words, and a storage unit that stores the plurality of words Among the plurality of words, a determination unit that determines whether there is a word that matches or is similar to a word included in the message received by the transmission / reception unit, and the acquisition unit according to a determination result of the determination unit Selecting either the photographed image acquired by the above or the substitute image stored in the storage means as the image transmitted by the transmission / reception means The communication terminal; and a-option means.
When the avatar image is stored in the control device 30, the control unit 11 of the mobile communication terminal 10 transmits designation data for designating the avatar image from the wireless communication unit 12, and the control device 30 transmits the designation data. The avatar image specified by is transmitted to the destination mobile communication terminal 10. In this case, the mobile communication terminal 10 includes an acquisition unit that acquires a captured image captured by the imaging unit, a transmission / reception unit that transmits / receives an image or data and a message, and the transmission / reception unit. Display means for displaying an image received by the means, storage means for storing a plurality of words, and words included in the message received by the transmission / reception means among the plurality of words stored by the storage means A determination unit that determines whether there is a matching or similar word, and a designation that specifies a captured image acquired by the acquisition unit or a substitute image that is a substitute for the captured image, according to a determination result of the determination unit A communication terminal comprising: selection means for selecting any of the data as an image or data transmitted by the transmission / reception means.

(6) Modification 6
In the above-described embodiment, videophone communication, that is, communication using an image and a voice message is performed. However, for example, characters (text messages) may be used instead of the voice message as in a so-called chat. In such a case, when the avatar displayed in the virtual space moves to the specific area described above, the avatar image displayed in the virtual space may be switched to the captured image. Note that the number of mobile communication terminals 10 that transmit and receive images and messages is not limited to two, and may be three or more.

(7) Modification 7
It is not necessary to instruct both of the mobile communication terminals 10 to switch between the avatar image and the captured image, and only one of them may be used. For example, if settings such as always displaying the captured image to the other party as the transmission destination or displaying the substitute image at all times because you do not want to display the captured image to the other party as the transmission destination, The device 30 switches and displays the avatar image and the captured image only on the counterpart mobile communication terminal 10 according to the elapsed time.

(8) Other Modifications The substitute image is not limited to the avatar image exemplified in the embodiment, as long as the image is a substitute for the captured image.
In the embodiment, when switching from a captured image to an avatar image, or when switching from an avatar image to a captured image, the user of the mobile communication terminal 10 confirms whether the image can be switched before switching. It was. This confirmation is not necessarily required, and the image may be switched immediately when the condition for switching the image is satisfied.
In the embodiment, the video phone is started by searching for the other party in the virtual space provided by the control device 30, but this is not necessarily required. The telephone number of the user who is the other party of the call or a user ID instead of it is provided to the user by some means, and the user makes a call using the telephone number, the user ID, etc. May be started.
In the embodiment described above, the mobile communication terminal 10 is a mobile phone, but may be other communication terminals. For example, it may be a portable communication terminal such as a PDA (Personal Digital Assistance) or a computer device such as a personal computer. When the communication terminal does not use a mobile communication network, another network such as the Internet may be used instead of the mobile communication network. In addition, the communication terminal does not need to incorporate a photographing unit, and may be configured such that photographed image data is supplied from an external video camera. This also applies to microphones and speakers.

  Further, the data transmitted by the mobile communication terminal 10A in step Sa1 includes data representing a telephone number. However, this data does not have to be a telephone number, and the mobile communication terminal 10 that is the communication source is uniquely identified. As long as the information can be identified. In this case, the control device 30 may make an inquiry to the service control station based on such information to obtain a telephone number.

The functions of the control device 30 may be distributed and provided to a plurality of devices. In short, a storage device that stores a plurality of words, an acquisition device that acquires and transmits the message that is transmitted and received between the first communication terminal and the second communication terminal that transmit and receive an image and a message and display the image; A determination device for determining whether or not there is a word that matches or is similar to a word included in a message acquired by the acquisition device among the plurality of words stored in the storage device; and the first communication terminal Alternatively, at least one of the second communication terminals can receive either one of the photographed image transmitted from the other communication terminal or the substitute image in place of the photographed image. A control device to be displayed according to the determination result may be provided in the mobile communication system.
The control device 30 is a separate device from the mobile communication network node device (service control station, switching office, etc.). However, the control device 30 is built in any one of the mobile communication network node devices. Also good.

  Various functions realized by the control device 30 can be provided as a single program. Therefore, the program can be provided in a form recorded on a recording medium, or can be provided by downloading from another control device 30 via a network such as the Internet.

1 is a diagram illustrating an overall configuration of a system according to an embodiment of the present invention. It is a block diagram which shows the structure of a control apparatus. It is a figure which shows the structure of the avatar ID conversion table which the control apparatus has memorize | stored. It is a figure which shows the structure of the distance management table which the control apparatus has memorize | stored. It is a block diagram which shows the structure of a mobile communication terminal. It is a figure which shows the operation element with which a mobile communication terminal is provided. It is a figure which shows the logical structure of each part implement | achieved by a mobile communication terminal. (A) is a figure which shows an example of an avatar image, (b) is a figure which shows an example of a picked-up image. It is a flowchart which shows the process which a mobile communication terminal performs. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays. It is a sequence chart which shows the process which a mobile communication terminal and a control apparatus perform. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays. It is the figure which illustrated the image which a mobile communication terminal displays.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10, 10A, 10B ... Mobile communication terminal, 11 ... Control part, 12 ... Wireless communication part, 13 ... Operation part, 14 ... Display part, 15 ... Audio | voice input / output part, 16 ... Imaging | photography part, 17 ... Multimedia processing part, DESCRIPTION OF SYMBOLS 20 ... Mobile communication network, 30 ... Control apparatus, 31 ... Control part, 33 ... Memory | storage part.

Claims (12)

Storage means for storing a plurality of words;
An acquisition means for acquiring the message transmitted and received between the first communication terminal and the second communication terminal for transmitting and receiving an image and a message and displaying the image;
A determination unit that determines whether there is a word that matches or is similar to a word included in the message acquired by the acquisition unit among a plurality of words stored by the storage unit;
Either one of the photographed image transmitted from the other communication terminal to the communication terminal of at least one of the first communication terminal and the second communication terminal or a substitute image in place of the photographed image And a control unit that displays the image according to the determination result of the determination unit. The control means includes
When transmission / reception of the image and the message is started between the first communication terminal and the second communication terminal, communication of at least one of the first communication terminal and the second communication terminal Display the backup image on your device,
After displaying the substitute image on the one communication terminal, among the plurality of words stored by the storage unit, there is a word that matches or is similar to the word included in the message acquired by the acquisition unit The control apparatus according to claim 1, wherein, when determined, the captured image is displayed on the one communication terminal. A history storage means for storing a history of transmission / reception of the image or the message made by the first communication terminal and the second communication terminal;
The control means includes
When transmission / reception of the image and the message is started between the first communication terminal and the second communication terminal in which transmission / reception history is not stored in the history storage unit, the first communication terminal or The first communication terminal and the second communication terminal in which the substitute image is displayed on at least one of the second communication terminals and the transmission / reception history is stored in the history storage unit When transmission / reception of the image and the message is started, the captured image is displayed on at least one of the first communication terminal and the second communication terminal. The control device according to claim 2. The control means includes
When transmission / reception of the image and the message is started between the first communication terminal and the second communication terminal, communication of at least one of the first communication terminal and the second communication terminal Display the captured image on the terminal,
When there is a word that matches or is similar to a word included in the message acquired by the acquisition unit among a plurality of words stored by the storage unit after the captured image is displayed on the one communication terminal. The control device according to claim 1, wherein, when determined, the substitute image is displayed on the one communication terminal. Each of the first communication terminal and the second communication terminal stores the substitute image displayed by the communication terminal of the other party that is the destination of the image or message from the own terminal,
The control means includes
When displaying the photographed image on at least one of the first communication terminal and the second communication terminal, the photographed image obtained by the other communication terminal and the message, Instruct the other communication terminal to send
When displaying the substitute image on at least one of the first communication terminal and the second communication terminal, the substitute image stored in the other communication terminal and the message are transmitted. The control device according to claim 1, wherein an instruction is given to the other communication terminal. Each of the first communication terminal and the second communication terminal stores a plurality of alternative images that can be displayed as alternatives to the captured image of the other party in the storage means of the own terminal,
The control means includes
When displaying the photographed image on at least one of the first communication terminal and the second communication terminal, the other communication terminal transmits the photographed image and the message obtained by photographing. Instruct the other communication terminal to
When displaying the substitute image on at least one communication terminal of the first communication terminal or the second communication terminal, the substitute image designated by the other communication terminal is notified to the one communication terminal. The one communication terminal is instructed to display the notified substitute image among the plurality of substitute images stored in the storage unit of the one communication terminal. Control device. Comprising a substitute image storage means for storing the substitute image;
The control means includes
When displaying the photographed image on at least one of the first communication terminal and the second communication terminal, the other communication terminal transmits the photographed image and the message obtained by photographing. Instruct the other communication terminal to
When displaying the substitute image on at least one communication terminal of the first communication terminal or the second communication terminal, the substitute image is read from the substitute image storage unit and is sent to the one communication terminal. The control device according to claim 1, wherein the control device transmits. The substitute image storage means stores the identifier assigned to the first communication terminal or the second communication terminal and the substitute image in association with each other,
When displaying the substitute image on at least one of the first communication terminal and the second communication terminal, the control means is associated with an identifier assigned to the other communication terminal. The control device according to claim 7, wherein the substitute image stored in the step is read from the substitute image storage unit and transmitted to the one communication terminal. A storage device for storing a plurality of words;
An acquisition device for acquiring the message, which is transmitted and received between the first communication terminal and the second communication terminal that transmit and receive an image and a message and display the image;
A determination device that determines whether there is a word that matches or is similar to a word included in a message acquired by the acquisition device among a plurality of words stored by the storage device;
Either one of the photographed image transmitted from the other communication terminal to the communication terminal of at least one of the first communication terminal and the second communication terminal or a substitute image in place of the photographed image A mobile communication system, comprising: a control device that displays a signal according to a determination result of the determination device. Obtaining means for obtaining a photographed image photographed by the photographing means;
Storage means for storing a substitute image in place of the captured image;
Transmitting and receiving means for transmitting and receiving images and messages to and from the communication terminal of the communication destination;
Display means for displaying an image received by the transmission / reception means;
Storage means for storing a plurality of words;
A judging means for judging whether or not there is a word that matches or is similar to a word included in a message received by the transmitting / receiving means among a plurality of words stored by the storage means;
A selection unit that selects, as an image to be transmitted by the transmission / reception unit, one of a captured image acquired by the acquisition unit or a substitute image stored in the storage unit according to a determination result of the determination unit; A communication terminal comprising: Obtaining means for obtaining a photographed image photographed by the photographing means;
A transmission / reception means for transmitting / receiving an image or data and a message to / from a communication terminal of a communication destination;
Display means for displaying an image received by the transmission / reception means;
Storage means for storing a plurality of words;
A judging means for judging whether or not there is a word that matches or is similar to a word included in a message received by the transmitting / receiving means among a plurality of words stored by the storage means;
Depending on the determination result of the determination means, either the photographed image acquired by the acquisition means or the designation data specifying the substitute image that is a substitute for the photographed image, the image transmitted by the transmission / reception means or A communication terminal comprising: selecting means for selecting as data. Storage means for storing a plurality of substitute images,
The display means displays, instead of the image received by the transmission / reception means, an alternative image specified by the designation data received by the transmission / reception means among the alternative images stored in the storage means. The communication terminal according to claim 11.

Images