JP5461651B2 - Service server device, service providing method, service providing program - Google Patents

Description

  The present invention relates to a service server device, a service providing method, and a service providing program that provide a service via communication, and in particular, a service server device, a service providing method, and a service providing that provide a service other than a call during a call using a telephone. Regarding the program.

Currently, translators and translation systems that automatically translate the content spoken by a speaker into an arbitrary language into other languages have been put into practical use. Conventional translators include a so-called single-person or face-to-face conversation type and a remote conversation type.
FIG. 8A is a diagram illustrating a translator for one person. In FIG. 8A, for example, the speaker inputs “How much is this book?” Into the translator in Japanese by voice. Then, the translation function of the translator translates this Japanese speech into English and outputs a synthesized speech of “How much is this book?”. A translator for one person can be used as a Japanese-English dictionary or used by a speaker to convey his / her intention to another person by letting others in the meeting listen to synthesized speech. Such a conventional translator is described in Non-Patent Document 1, for example.

  FIG. 8B is a diagram illustrating a remote conversation type translator. The translator shown in FIG. 8B has a telephone voice communication function and a translation function. In FIG. 8 (b), for example, a Japanese speaker is voice-inputted to the translation machine in his native language, “Please make a reservation with three people”. Then, the translation function of the translator translates this Japanese speech into English and outputs a synthesized speech of “Please make a reservation for 3 people”. A remote-conversation type translator allows a speaker to communicate in his / her own language while speaking in his / her native language. Such a conventional translator is described in Non-Patent Document 2, for example.

  Further, as a remote conversation type translator, there is one described in Patent Document 1. The telephone system described in Patent Document 1 can not only translate the contents of the telephone and transmit it to the called party, but can also transmit the call voice of the speaker to the called party. For this reason, the invention described in Patent Document 1 can give the user a feeling as if he / she is talking to the other party via an interpreter.

Japanese Patent No. 3741025

andro navi, [online], “iTranslator for Android-high-performance translation app that supports 20 languages-” [searched on May 13, 2011], Internet <URL: http://andronavi.com/2010/ 09/40880> iplatform. org, [online], Sekaiphone [searched on May 13, 2011], Internet <URL: http://www.iplatform.org/>

  However, in the technique of Non-Patent Document 2 above, since the speech translated from the content spoken by the speaker is transmitted only to the call destination, the speaker can understand how the content spoken by himself / herself is recognized and translated. I can't know. Further, even in the invention described in Patent Document 1, since the speech translated from the content spoken by the speaker and the call speech of the speaker are transmitted only to the other party, the content spoken by the speaker is transmitted to the system. I don't know how it was translated. Furthermore, in the invention described in Non-Patent Document 1, although the content entered by speech itself can be confirmed by converting it into text, it is originally a translation machine for one person, so the content entered by speech itself is confirmed. However, this content cannot be transmitted.

For this reason, when the content of a speaker's call is translated according to any of the techniques of Patent Literature 1, Non-Patent Literature 1, and Non-Patent Literature 2 described above, how the content spoken by the speaker is translated. It is difficult to recognize and correct errors.
The present invention has been made in view of the above points, and is a service server device and service that can realize a service that can easily cope with correction of translation errors performed based on the content of a speaker's call An object is to provide a providing method and a service providing program.

In order to solve the above-described problems, the service server device of the present invention (for example, the service server device 200 shown in FIG. 2) is connected to a plurality of mobile phones that perform voice communication with each other by voice, and voice performed between the mobile phones. A service server device that provides a service by another task that is a task executed by an application different from a task executed by an application related to communication, the information specifying the mobile phone and a user using the mobile phone and a subscriber information storage unit the subscriber information including at least is registered information on the originating terminal is one of a plurality of the mobile phone, the plurality of the mobile phone is added predetermined information when a call is using the telephone number of the receiving terminal is at least one, from the subscriber information storage unit It acquires subscriber information is recorded, if the acquired information identifying at least one of said originating terminal and said destination terminal to said subscriber information is included, during the voice communication instruction receiving section to start the row cormorants application execution of the further tasks (for example service activation unit 250 shown in FIG. 2), the recording unit to record in the voice communication between a plurality of the mobile telephone, the voice of the speaker (eg, telephone / call control enabler 201 of FIG. 2) and, if the application for execution of the another task by the instruction receiving unit is started, based on the sound recorded by the recording unit, the speaker of the mobile phone and the task execution unit to the other mobile telephone to voice communication to reflect the information about the user included in the subscriber information perform the specific task (e.g., FIG. 2 The speech recognition enabler 203, the text translation enabler 204, the speech synthesis enabler 202), the text data obtained by executing the separate task by the task execution unit, and the separate task being executed by the task execution unit. And a task data creation unit (for example, the speech recognition enabler 203, the text translation enabler 204, and the speech synthesis enabler 202 shown in FIG. 2) that perform task communication. to the mobile phone of the speaker, providing the task data, to the portable telephone to the cellular telephone and the voice communication of the speaker, the task data providing unit for providing the task data together with audio of the speaker (e.g. And a communication control unit 224) shown in FIG. And

The task execution unit may start the execution of the separate task when the sound recorded in the recording unit is a sound having a predetermined meaning.
In addition, it is preferable that the task execution unit starts the execution of the separate task when the signal output by the operation of the mobile phone is a predetermined signal.

The service providing method of the present invention is an operation executed by an application that is connected to a plurality of mobile phones that perform voice communication with each other by voice and that is different from a task executed by an application related to voice communication performed between the mobile phones. A service providing method for providing a service in a service server device for providing a service according to another task, wherein subscriber information including at least information for identifying the mobile phone and information on a user who uses the mobile phone is registered. a subscriber information storing step of, the calling terminal is one of a plurality of the mobile phone, at least is one telephone number of the receiving terminal of the plurality of mobile phone-determined information is added in advance when a call is using, the registered in the subscriber information registration step subscriber Gets the broadcast, if the acquired information identifying at least one of said originating terminal and said destination terminal to said subscriber information is included, the execution of said other tasks during the voice communication an instruction receiving step of activating a row cormorants application, in the speech communication between a plurality of the mobile phone, and recording step for recording the voice of the speaker, application for execution of the another task in the instruction receiving step is activated If so, based on the voice recorded in the recording step, the separate task is executed by reflecting information about the user included in the subscriber information of another mobile phone that performs voice communication with the speaker's mobile phone. A task execution step, and text data obtained by executing the other task in the task execution step; The audio data obtained by the separate tasks are executed by the task execution step, the task data generating step of generating a task data including, to the mobile phone of the speaker is performing voice communication, the task providing data to said mobile telephone to the cellular telephone and the voice communication of the speaker, characterized in that it comprises a and a providing step of providing the task data together with audio of the speaker.

The service providing program of the present invention is an operation executed by an application that is connected to a plurality of mobile phones that communicate with each other by voice and that is different from a task executed by an application related to voice communication performed between the mobile phones. A service providing program that is executed in a service server device that provides a service according to another task, the computer including subscriber information including at least information for identifying the mobile phone and information on a user who uses the mobile phone a subscriber information storage function of registering, calling terminal is one of a plurality of the mobile phone, the telephone number of the least is one called terminal of the plurality of mobile phone-determined information is added in advance when a call is using, Noboru in the subscriber information registration step It has been acquired subscriber information, if the acquired information identifying at least one of said originating terminal and said destination terminal to said subscriber information is included, said other during the voice communication performs an instruction receiving function for starting the execution of the task line cormorants application, in the speech communication between a plurality of said portable telephone, and a recording function to record the voice of the speaker, the execution of the specific task by said instruction reception function When the application is started, based on the voice recorded by the recording function , the information about the user included in the subscriber information of another mobile phone that performs voice communication with the speaker's mobile phone is reflected, and A task execution function for executing another task, text data obtained by executing the another task by the task execution function, and the task And audio data obtained by the separate task is performed by the execution function, the task data creation function to create a task data including, to the mobile phone of the speaker is performing voice communication, the task data providing, in the mobile telephone to the cellular telephone and the voice communication of the speaker, characterized in that to realize a providing function of providing the task data together with audio of the speaker.

According to the present invention, task data including text and voice is created by executing the task and transmitted to each of the voice communication terminal devices that are making a call. Can also be received. For this reason, it becomes easy to deal with an error or correction of a task executed based on the content of the speaker's call.
Moreover, according to this invention, task data and a speaker's audio | voice can be transmitted to a telephone call partner. For this reason, when the task data includes translated speech, the translated speech of the speaker can be transmitted to the call partner together with the task execution result. In the present invention, since the task is executed by an application different from the application related to voice communication, the task voice is executed by using the existing voice application and the task executing application. The result can be sent to the other party. The present invention as described above can provide a call service with a sense of presence such that a user is talking face-to-face while using an existing application.

It is a figure for demonstrating the outline | summary of the service used as the premise of embodiment of this invention. It is a block diagram of the service server apparatus of embodiment of this invention. It is a diagram for explanation of the procedure of processing performed by the service server apparatus shown in FIG. It is a figure for demonstrating concretely the CS network and PS network which connect the service server apparatus shown in FIG. 2, FIG. 3, an originating communication terminal device, and a destination communication terminal device. It is a figure for demonstrating concretely the PS network which connects the service server apparatus shown in FIG. 2, FIG. 3, an originating communication terminal device, and a destination communication terminal device. It is a figure for demonstrating the service provision method of embodiment of this invention. It is a figure explaining the example which applies this invention to the application which adjusts a user's schedule. FIG. 8A is a diagram for explaining the configuration of a general translator, and FIG. 8B is a diagram for explaining the configuration of a general translation system.

  Hereinafter, embodiments of a service server device, a service providing method, and a service providing program according to the present invention will be described with reference to the drawings. In each drawing referred to below, components similar to those in other drawings are denoted by the same reference numerals, and a part of the description may be omitted.

[Communication Concierge]
FIG. 1 is a diagram for explaining an outline of a service that is a premise of the present embodiment. This service is developed to support communication (hereinafter referred to as “call”) by a communication terminal device having a call function, and to realize communication such as meeting directly. The service that is the premise of the present embodiment is hereinafter also referred to as a CC (communication concierge) service.

  The CC service is a service provided to communication terminal apparatuses c1 to c5 connected to the network N as shown in FIG. Communicate either a telephone having a call function, a communication terminal device (for example, a smartphone) that can be connected to the network N and can send and receive text by packet communication, or a communication terminal device that can send and receive both voice and text. It can be used as the terminal devices c1 to c5. The communication terminal devices c1 to c5 may have a function of transmitting / receiving not only text but also still images, moving images, and music.

  In the present embodiment, it is assumed that a voice call is performed between a user of at least one of the communication terminal devices c1 to c5 connected to the network N and a user of at least one other device. ing. A user who is in a call using the telephone can activate another task and execute the task during the call. Information obtained by executing the activated task can be received by the telephone in communication or another communication terminal device corresponding to the telephone.

  In order to realize the present embodiment, a service server device that provides a service by a task different from a call during a call using a telephone is necessary. A service server device for realizing the present embodiment includes: translation of a call content; synthesis speech of a call content after translation; generation of data (hereinafter also referred to as task data) by inserting synthesized speech into the call speech; The translated call contents are converted into text, and the task data and the text data (hereinafter also referred to as text data) are linked and transmitted to the telephone or the like during the call.

Hereinafter, the service server device for realizing the present embodiment will be described more specifically.
[Service server device]
(I) Function FIG. 2 is a block diagram illustrating a configuration example of the service server device according to the present embodiment. 2 is configured to be able to communicate with an enabler group 261 including a plurality of enablers. The enabler is driver software for operating a computer, and refers to software (application) designed for each purpose.

  The enabler group 261 includes an enabler group 205 for a telephone function and an enabler group 206 added to realize a translation function that is a function unique to the present embodiment. The enabler group 205 includes a session cooperation enabler 218, a data storage enabler 207, a device management control enabler 208, a presence enabler 209, a telephone directory enabler 210, an SNS (Social Networking Service) enabler 211, a location information detection enabler 262, and telephone / call control. An enabler 201, a messaging enabler 212, and the like are included.

  The enabler group 206 includes a point management enabler 219, a community management enabler 213, an information management enabler 214 that manages predetermined information according to the application, a voice synthesis enabler 202, an image / video synthesis / editing enabler 215, and a font conversion enabler 216. , A voice recognition enabler 203, an image / video recognition enabler 217, a text translation enabler 204, and the like.

  The enabler group 205 for the telephone function is a set of enablers related to voice communication. The enabler group 206 is a set of enablers different from the application related to communication by voice. That is, in this specification, the enabler included in the enabler group 205 is a task executed by an application related to voice communication, and the enabler included in the enabler group 206 is a task executed by an application related to voice communication. Is another task that is an operation performed by another application.

  The enabler groups 205 and 206 included in the enabler group 261 constitute an environment for providing a service on the network side, a so-called service enabler network (hereinafter referred to as SEN) 2. SEN2 can provide various services in cooperation with a plurality of Internet players P that can provide a plurality of services including a translation service.

  In this embodiment, in order to execute a task related to translation, the service server apparatus 200 acquires the telephone / call control enabler 201 of the enabler group 205 and stores it in the service application unit. Also, the service server device 200 stores the speech synthesis enabler 202, the speech recognition enabler 203, and the text translation enabler 204 from the enabler group 206 in the service application unit. As described above, in this embodiment, all service application units include the service activation unit 250. As a result, in this embodiment, the service application unit becomes the service application unit 225 that provides a service related to translation.

The service server device 200 can select and obtain an enabler necessary for executing a task according to the type of service. The enabler acquired by the service server device 200 is stored for each service in the service server device 200 as a service application (service application software) unit described later.
With such a configuration, the service server apparatus 200 according to the present embodiment can construct an application necessary for a service by combining existing enablers. For this reason, according to the present embodiment, it is not necessary to newly develop a dedicated application, and time and cost for developing the application do not occur. Furthermore, according to the present embodiment, since a service can be provided using an existing application with a proven track record, it is possible to provide a service with higher reliability compared to developing a new application.

  In the present embodiment, it is assumed that the calling communication terminal device 241a and the called communication terminal device 241b are provided with a service by the service server device 200 while talking to each other. The service server device 200 executes a task when providing a service to the calling communication terminal device 241a and the called communication terminal device 241b. The outgoing communication terminal device 241a and the incoming communication terminal device 241b in the present embodiment are, for example, cellular phones that have both a telephone function and a function capable of transmitting and receiving text, still images, moving images, and the like.

  The service server device 200 has an existing configuration for controlling a telephone call as well as a service application unit having an enabler corresponding to a service. As a configuration for controlling a call, a subscriber profile storage unit 221 that stores data relating to a CC service subscriber (that is, a user who receives the CC service, hereinafter simply referred to as a “user”), a request for providing the CC service There are an authentication permission unit 222 that authenticates and authorizes, a charging processing unit 223 that performs processing related to charging imposed on a call, and a communication control unit 224 that controls call and packet communication. The communication control unit 224 also has a function as a providing unit that provides task data to the originating communication terminal device 241a and the destination communication terminal device 241b, respectively.

In the subscriber profile storage unit 221, the language type used by the user is registered in advance together with the user's name and the telephone number of the telephone being used. In addition, MSISDN (Mobile Subscriber ISDN Number: telephone number registered in the SIM card of the telephone) included in the telephone data communication session is registered in the subscriber profile storage unit 221 as information for specifying the telephone. ing. Further, in order to associate the devices such as the tablet PC and the photo frame with the telephone, the identifiers of the devices may be registered in the subscriber profile storage unit 221 in association with the telephone number of the corresponding telephone.

  The service server device 200 may include a plurality of service application units 225, 226, and 227. In the first embodiment, the plurality of service application units 225, 226, and 227 may be storage areas that can store, for example, a plurality of enablers. In this case, the storage area can be any of the service application units 225, 226, and 227 by the stored enabler. In addition, a service activation unit 250 is stored in a storage area serving as a service application unit of the present embodiment. The service activation unit 250 has a function of activating an enabler to receive an instruction transmitted from the originating communication terminal device 241a via the IP network 230 and provide a service, for example.

  In the present embodiment, when the user operates the originating communication terminal device 241a, the service activation unit 250 activates the enabler stored in the service application unit 225. When the enabler stored in the service application unit 225 is activated, execution of the task is started. The activated telephone / call control enabler 201 records the voice of the speaker (hereinafter referred to as “call voice”) during communication between the calling communication terminal device 241a and the called communication terminal device 241b. Note that the recording of the call voice may be performed by controlling the recording of a well-known answering machine provided in the telephone / call control enabler 201 and using the call voice recording function of the answering machine.

The voice recognition enabler 203 creates text data from the recorded voice. For example, the voice recognition enabler 203 analyzes the recorded voice by morphological analysis, and converts the voice into text data.
The text translation enabler 204 creates translated text data by translating the text data created by the speech recognition enabler 203 into the language used by the user of the destination communication terminal device 241b.

The speech synthesis enabler 202 creates speech data to be read out by synthesized speech by converting the translated text data created by the text translation enabler 204 into speech represented by speech.
In the present embodiment, the voice data created by the voice synthesis enabler 202 is provided to the calling communication terminal device 241a and the called communication terminal device 241b together with the call voice. In this embodiment, the text data created by the speech recognition enabler 203 and the translated text data generated by the text translation enabler 204 are also provided to the calling communication terminal device 241a and the called communication terminal device 241b. In the present embodiment, the voice data, the text data, and the translated text data are also referred to as task data.

  As described above, the speech recognition enabler 203, the text translation enabler 204, and the speech synthesis enabler 202 are based on the voice recorded by the telephone / call control enabler 201 when a task execution instruction is received by the service activation unit 250. A task is executed, and task data is created by executing the task. This task data is provided to the calling communication terminal device 241a and the called communication terminal device 241b.

In the service server device of the present embodiment configured as described above, the service activation unit 250 functions as an instruction receiving unit. Further, the telephone / call control enabler 201 functions as a recording unit, and the speech recognition enabler 203, the text translation enabler 204, and the speech synthesis enabler 202 function as a task execution unit and a task data creation unit.
In the present embodiment, the service activation unit 250 does not immediately activate each enabler when the originating communication terminal device 241a and the destination communication terminal device 241b instruct the provision of the service, but from the subscriber profile accumulation unit 221 in advance. You may make it acquire the registered subscriber information. When the subscriber information includes information for specifying at least one of the calling communication terminal device 241a and the called communication terminal device 241b, an instruction from the calling communication terminal device 241a and the called communication terminal device 241b is received. Also good.

Further, in the present embodiment, when an instruction to provide service, the mobile phone telephone number, the start conditions of the service activation unit 250 so as to transmit by adding information such as a specific number set in advance It may be set. When the activation conditions are set in this way, the service activation unit 250 makes a call using at least one of the calling communication terminal device 241a and the called communication terminal device 241b using a telephone number to which predetermined information is added. In this case, an instruction from the calling communication terminal device 241a and the called communication terminal device 241b is received.

  Further, the present embodiment is not limited to the case where the service activation unit 250 activates each enabler by a signal output by an operation of the calling communication terminal device 241a and the called communication terminal device 241b. For example, when a call voice recorded by the telephone / call control enabler 201 is input, the service activation unit 250 may activate each enabler and start executing a task. When the service activation unit 250 is configured in this way, a word that can identify an application, for example, a word such as “translation start” is set in advance. Then, as a result of morphological analysis of the call voice recorded by the voice recognition enabler 203, the service activation unit 250 activates each enabler and executes a task only when a preset wording is included in the call voice. .

  In the present embodiment described above, the calling communication terminal device 241a and the called communication terminal device 241b are not limited to mobile phones capable of transmitting text. When the calling communication terminal device 241a and the called communication terminal device 241b are telephones having only a call function, a task execution result corresponding to a tablet PC 242a, 242b associated with the telephone in advance by a request from a telephone having only a call function, You may make it transmit to the photo frames 243a and 243b.

(Ii) Processing Procedure FIG. 3 is a diagram for explaining a processing procedure performed by the service server apparatus shown in FIG.
In the service application unit 225 shown in FIG. 2, the call voice recording is performed by the telephone / call control enabler 201 shown in FIG. 2 (step S311). The recording of the call voice is performed with one unit of the call voice that is emitted from a state where no voice is recorded (hereinafter also referred to as a silent recording state) to the next silent recording state. Next, the speech is analyzed by the speech recognition enabler 203 and analyzed. The analysis is performed for each call voice recorded as one unit. Then, the call voice is converted into text data according to the analysis result (step S312).

  Next, in the service application unit 225, the text data converted in step S312 by the text translation enabler 204 is translated text data in a language registered in the subscriber profile storage unit 221 in advance by the user of the destination communication terminal device 241b. (Step S313). The translated text data after translation is converted into synthesized speech and becomes speech data (step S314). Furthermore, voice data is inserted into the call voice (step S315).

  Here, “inserting voice data into the call voice” means determining the timing of sending the voice data in accordance with the call voice. By determining the transmission timing of the voice data in this way, the call voice is transmitted to the calling communication terminal device 241a and the called communication terminal device 241b through the CS (Circuit Switched) network as usual. Voice data is also transmitted to the calling communication terminal device 241a and the called communication terminal device 241b by a data communication session via the CS network, but may be transmitted via the PS network (Packet Switched). . Further, text data obtained by translating the call sound properties into text is transmitted using the PS network. In FIG. 3, the CS network and the PS network are denoted by reference numeral 320.

However, the present embodiment is not limited to one that transmits call voice over the CS network. For example, the call voice and voice data may be transmitted to the calling communication terminal device 241a and the called communication terminal device 241b via the PS network.
In the present embodiment, as described above, task data is created by recording, recognizing and translating communication speech uttered between the silent recording state and the next silent recording state as a unit, and each time the outgoing communication is performed. By transmitting to the terminal device 241a and the destination communication terminal device 241b, the call voice and task data can be synchronized and transmitted to the source communication terminal device 241a and the destination communication terminal device 241b.

  In the present embodiment, in order to avoid a sense of incongruity in the call, the call voice is transmitted to the communication destination terminal device 241a and the destination communication terminal device 241b in real time as usual. For this reason, when voice data is inserted into the call voice, the user may hear the voice based on the voice data superimposed on the call voice. However, when talking through an interpreter, it is generally considered that people tend to refrain from speaking while the interpreter is speaking. For this reason, it is unlikely that the voice based on the voice data and the voice of the call overlap to hinder the conversation.

In the present embodiment, both the calling communication terminal device 241a and the called communication terminal device 241b access the service application unit 225 to acquire text data, translated text data, and voice data. In this way, a method of connecting the originating communication terminal device 241a and the destination communication terminal device 241b via the service application unit 225 is called bridge connection.
Further, the service server apparatus 200 shown in FIG. 1 can identify the telephone line of the calling communication terminal apparatus 241a and the telephone line of the called communication terminal apparatus 241b by a known method. Therefore, it is possible to distinguish the uplink u1 and downlink d1 of the calling communication terminal device 241a, the uplink u2 and downlink d2 of the called communication terminal device 241b, and appropriately translate the user's utterance and transmit it to the call partner can do.

  According to the present embodiment described above, it is possible to provide call voice, synthesized voice data, text data, and translated text data to the calling communication terminal device 241a and the called communication terminal device 241b. For this reason, the user can easily recognize how the content uttered by the user is recognized by the service server apparatus 200, and can take corrective action such as quickly correcting any mistranslation. In addition, since the user can hear both the call voice of the other party and the voice data obtained by translating the call voice, it is possible to realize a conversation with a sense of presence as if the user was talking through an interpreter. .

(Iii) CS Network and PS Network FIG. 4 shows a specific example of the CS network 404 and the PS network 405 that connect the service server apparatus 200 shown in FIGS. 2 and 3 with the originating communication terminal apparatus 241a and the destination communication terminal apparatus 241b. It is a figure for demonstrating.
The calling communication terminal device 241a makes a call to an MPN (Media Processing Node) 408. This call signal reaches the CS network 404 and the existing telephone system 401. The existing telephone system device 401 refers to, for example, CSN (authentication device, home agent for realizing handover, DHCP server, etc.), ASN (wireless base station, etc.). In the existing telephone system 401, non-billing processing and special number routing processing are executed for outgoing calls. The non-billing process is a process for preventing the billing process for a general call. Further, special number routing means changing a communication path to a communication path in which processing different from the original processing is performed when a call is made to a predetermined telephone number (special number).

Next, the call signal reaches the service server device 200 of the present embodiment from the existing telephone system device 401. The service server device 200 includes a communication control unit 224 that controls communication of the telephone set shown in FIG. 1, a subscriber profile storage unit 221, an authentication permission unit 222, and a billing processing unit 223. Service restriction from other networks and linkage between voice and screen (linkage between voice data and text data, etc.) are performed.
The linkage between voice data, text data, and translated text data can be realized as follows.

  That is, when a telephone that can receive provision of this service enters a call state, the communication control unit 224 of the service server apparatus 200 executes a data communication session including an MSISDN that matches the MSISDN included in the signal related to the telephone call. Start monitoring. When a data session that matches the MSISDN included in the signal related to the call is performed, it is determined that the call and the data session should be linked in the CC service. According to such a method, when the call voice, voice data, text data, and translated text data are transmitted to one communication terminal device, the call voice transmitted by the CS network 404 and the PS network 405 are transmitted. Can link with task data.

Further, as described above, in the present embodiment, the call voice can be transmitted through the PS network 405 in the same manner as the voice data. FIG. 5 shows a configuration for transmitting a call voice together with voice data and the like via the PS network 405.
In the first embodiment, as described above, the subscriber profile storage unit 221 shown in FIG. 2 associates and registers a telephone having only a call function and a communication terminal device capable of transmitting text or the like. . In such a case, when a call is made with the telephone number of the telephone, the communication control unit 224 may transmit a URL that can acquire task data to the address of the registered communication terminal device.

The call voice corresponding to the outgoing call is recorded by the telephone / call control enabler 201 in the MPN 408 and passed to the service server apparatus 200.
In the service server device 200, the application unit 225 creates voice data, text data, and translated text data based on the recorded call voice. The voice data is transmitted to the destination communication terminal device 241b via the CS network. Note that a PS network can also be used for transmitting voice data. Also, text data and translated text data are transmitted to the destination communication terminal device 241b via the PS network 405 from an existing Web system device 402 (gateway or the like) or xGSN (x-GPRS Support Node) / EPC (Evolved Packet Core). Is done.

(Iv) Service Providing Method FIG. 6 is a diagram for explaining a service providing method realized by the service server device 200 of the present embodiment. In FIG. 6, data communication using HTTP (Hypertext Transfer Protocol) is indicated by a broken line, and a telephone call is indicated by a solid line.
In the calling communication terminal device 241a, for example, the user activates an application for realizing the service providing method of the present embodiment, and the other party's telephone number is input (step S501). By such processing, the address of the calling communication terminal device 241a is transmitted from the calling communication terminal device 241a to the service server device. Then, the service server apparatus 200 receives an instruction to execute the task during communication (call) on the condition that other necessary requirements are satisfied, and starts the translation service (step S502).

The service server device 200 establishes a communication line between the calling communication terminal device 241a and the called communication terminal device 241b (step S503). Then, the service server apparatus 200 records the call voice transmitted from the line of the calling communication terminal apparatus 241a (step S504) and sends it to the line of the called communication terminal apparatus 241b (step S505).
When a silent recording state in which no call voice is transmitted from the line of the calling communication terminal apparatus 241a is entered, the service server apparatus 200 once ends the recording of the call voice transmitted from the line of the calling communication terminal apparatus 241a (step S506). . The recorded call voice is converted into text, translated, and synthesized voice is synthesized by the voice recognition enabler 203, the text translation enabler 204, and the voice synthesis enabler 202 shown in FIGS. 2 and 3 (step S507). Create voice data, text data, and translated text data.

  The service server device 200 according to the present embodiment transmits the voice data to both the originating communication terminal devices 241a and 241b, and the text data and the translated text data are both the originating communication terminal device 241a and the terminating communication terminal device 241b. (Steps S508 to 511). Next, when the user of the destination communication terminal apparatus 241b speaks, the service server apparatus 200 performs the above operation on the voice of the user's call transmitted from the telephone line of the destination communication terminal apparatus 241b. Note that, when one of the two users speaks while the other speaks, in the present embodiment, the above processing is performed on the call voice of the user who is recording the call voice, and the other user The content of the utterance should not be processed.

Note that the present embodiment is not limited to the configuration described above. That is, in the present embodiment described above, the CC service related to translation is executed for a call between two users. However, this embodiment is not limited to the one intended for a call between two people, and can also be applied to a conference call or the like in which three or more people make a call.
Furthermore, the present invention is not limited to the configuration applied to the application relating to translation of the present embodiment. FIG. 7 is a diagram illustrating an example in which the service server device, the service control method, and the service control program of the present invention are applied to an application that adjusts a user's schedule.

  In the example shown in FIG. 7, the service server device provides a service by a scheduler task different from the voice communication during the communication of the calling communication terminal device 641a and the called communication terminal device 641b. In such an example, the user of the calling communication terminal device 641a instructs the service server device by operating the calling communication terminal device 641a to execute the scheduler task during the call. When the service server device accepts the instruction, the call voice of the user of the calling communication terminal device 641a and the called communication terminal device 641b is recorded during outgoing communication.

  In the example shown in FIG. 7, the service server device acquires call voices from the telephone lines of the calling communication terminal device 641a and the called communication terminal device 641b and records them respectively. The voice recognition unit shown in FIG. 2 detects that a voice meaning “schedule adjustment” is included in the call voice by performing morphological analysis on the recorded call voice. Since “schedule adjustment” is included in the call voice, the task of extracting the date and time when the user's schedules of the calling communication terminal device 641a and the called communication terminal device 641b are both free is executed. Judge that it was instructed.

Then, for example, the information management enabler 214 shown in FIG. 2 functions as a schedule information management unit, and acquires schedule data of users registered in the calling communication terminal device 641a and the called communication terminal device 641b. The information management unit 214 extracts from the acquired schedule data the date on which the schedules of the two users are not registered and its time zone. The extracted date / time is transmitted as image data including text data to both the calling communication terminal device 641a and the called communication terminal device 641b.
According to such a configuration, it is possible to easily and definitely adjust the schedule for the user of the originating communication terminal device 641a and the destination communication terminal device 641b, and to have a sense of presence similar to the case where the schedule adjustment is performed directly face-to-face. Can be provided.

(V) Service providing program In the service server apparatus described above, a service providing program for providing a service by a task different from voice communication is executed during communication between voice communication terminal devices capable of voice communication. Is done. The service providing program includes an instruction receiving function for receiving an instruction to execute another task during voice communication on a computer, and a recording function for recording a speaker's voice during voice communication between the plurality of voice communication terminal devices. And when the instruction is received by the instruction receiving function, the task execution function for executing the task based on the voice recorded by the recording function and the task being executed by the task execution function are obtained. A task data creation function for creating task data including text or voice, and a voice communication terminal device performing voice communication, or a text communication terminal device associated with a voice communication terminal device and capable of text communication And a function for providing task data.

  The present invention can provide a service providing apparatus, a service providing method, and a service providing program that are optimal for supporting communication using a smartphone or the like.

DESCRIPTION OF SYMBOLS 200 Service server apparatus 201 Call control enabler 202 Speech synthesis enabler 203 Speech recognition enabler 204 Text translation enabler 205, 206, 261 Enabler group 208 Device management control enabler 209 Presence enabler 210 Phone book enabler 211 SNS enabler 212 Messaging enabler 213 Community management enabler 214 Information management enabler 215 Editing enabler 216 Font conversion enabler 217 Image / video recognition enabler 218 Session cooperation enabler 219 Point management enabler 221 Subscriber profile storage unit 222 Authentication permission unit 223 Charge processing unit 224 Communication control unit 225, 226, 227 Service application unit 230 Network 241a, 641a Originating communication terminal device 241b, 641b Arrival communication terminal device

Claims (5)

Connected to a plurality of mobile phones that communicate with each other by voice, and provides a service by another task that is a task executed by an application different from a task executed by an application related to voice communication performed between the mobile phones A service server device,
A subscriber information storage unit in which subscriber information including at least information for specifying the mobile phone and information on a user who uses the mobile phone is registered;
If the calling terminal is one of a plurality of said portable telephone has a call using at least is one telephone number of the receiving terminal of the plurality of mobile phone-determined information is added in advance, When the subscriber information registered from the subscriber information storage unit is acquired, and the acquired subscriber information includes information specifying at least one of the transmitting terminal and the receiving terminal , an instruction receiving section to start the row cormorants application execution of the specific tasks during the voice communication,
A recording unit for recording a voice of a speaker during voice communication between the plurality of mobile phones;
The subscriber information of another mobile phone that performs voice communication with the speaker's mobile phone based on the voice recorded by the recording unit when an application that executes the different task is started by the instruction receiving unit. A task execution unit that executes the separate task by reflecting information about the user included in
Task data creation for creating task data including text data obtained by executing the another task by the task execution unit and voice data obtained by executing the other task by the task execution unit And
To the mobile phone of the speaker is performing voice communication, providing the task data, to the portable telephone to the cellular telephone and the voice communication of the speaker, providing the task data together with audio of the speaker A task data provider,
A service server device comprising: The task execution unit
2. The service server device according to claim 1, wherein when the voice recorded in the recording unit is a voice having a predetermined meaning, execution of the another task is started. The task execution unit
2. The service server device according to claim 1, wherein when the signal output by operating the mobile phone is a predetermined signal, the execution of the another task is started. Connected to a plurality of mobile phones that communicate with each other by voice, and provides a service by another task that is a task executed by an application different from a task executed by an application related to voice communication performed between the mobile phones A service providing method for providing a service in a service server device, comprising:
A subscriber information accumulating step for registering subscriber information including at least information for specifying the mobile phone and information on a user who uses the mobile phone;
If the calling terminal is one of a plurality of said portable telephone has a call using at least is one telephone number of the receiving terminal of the plurality of mobile phone-determined information is added in advance, When the subscriber information registered in the subscriber information registration step is acquired, and the acquired subscriber information includes information specifying at least one of the calling terminal and the receiving terminal, an instruction receiving step of activating a row cormorants application execution of the specific tasks during the voice communication,
A recording step of recording the voice of the speaker during voice communication between the plurality of mobile phones;
When an application for executing the other task is started in the instruction receiving step, the subscriber information of another mobile phone that performs voice communication with the speaker's mobile phone based on the voice recorded in the recording step A task execution step of executing the separate task by reflecting information on the user included in
Task data creation for creating task data including text data obtained by executing the another task in the task execution step and voice data obtained by executing the other task in the task execution step Steps,
To the mobile phone of the speaker is performing voice communication, providing the task data, to the portable telephone to the cellular telephone and the voice communication of the speaker, providing the task data together with audio of the speaker Providing step;
A service providing method comprising: Connected to a plurality of mobile phones that communicate with each other by voice, and provides a service by another task that is a task executed by an application different from a task executed by an application related to voice communication performed between the mobile phones A service providing program executed in the service server device,
On the computer,
A subscriber information storage function for registering subscriber information including at least information specifying the mobile phone and information on a user who uses the mobile phone;
If the calling terminal is one of a plurality of said portable telephone has a call using at least is one telephone number of the receiving terminal of the plurality of mobile phone-determined information is added in advance, When the subscriber information registered in the subscriber information registration step is acquired, and the acquired subscriber information includes information specifying at least one of the calling terminal and the receiving terminal, an instruction receiving function for starting the line cormorants application execution of the specific tasks during the voice communication,
A recording function for recording the voice of the speaker during voice communication between the plurality of mobile phones;
The subscriber information of another mobile phone that performs voice communication with the speaker's mobile phone based on the voice recorded by the recording function when an application that executes the different task is started by the instruction receiving function. A task execution function for executing the different task by reflecting information on the user included in
Task data creation for creating task data including text data obtained by executing the another task by the task execution function and voice data obtained by executing the other task by the task execution function Function and
To the mobile phone of the speaker is performing voice communication, providing the task data, to the portable telephone to the cellular telephone and the voice communication of the speaker, providing the task data together with audio of the speaker Provided functions,
A service providing program characterized by realizing the above.

Images