JP5220912B2 - Character information display device with speech synthesis function and control method thereof - Google Patents

Description

  The present invention relates to a character information display device with a voice synthesis function having a function of reading out a displayed item by voice and a control method thereof.

In recent years, there has been proposed a mobile phone that announces a function name or the like set by a key operation corresponding to the key operation as a mobile terminal device (see, for example, Patent Document 1).
The mobile phone includes a plurality of key operation units, a control unit configured to set a function corresponding to the key operation among a plurality of functions provided in the telephone according to one or more key operations of the key operation unit, and a key operation. And a voice synthesizer that outputs a function name set in conjunction with the voice.

  In addition, as a system that employs a voice output function, an e-mail system has been proposed in which a sender can select a sound quality used for text-to-speech on the receiving side when transmitting text by e-mail ( For example, see Patent Document 2).

Japanese Patent Laid-Open No. 11-252216 JP 2004-185055 A

  By the way, in the portable terminal device having the speech reading function as described above, it is realized by notifying a text to be read out to a reading engine (control unit or speech synthesis unit).

However, a browser such as the Internet notifies the mobile terminal device of drawing information for displaying a character string, but does not notify a character string for reading.
The drawing information is notified after the character string is finely divided and cannot be notified to the reading engine as it is.
In addition, since the order in which the character strings are notified is not necessarily notified from the upper part where they are displayed, if the reading is performed in the order in which they are notified, an appropriate sentence will not be obtained.
That is, depending on the drawing style, even if the character strings are on the same line, the coordinate values are notified with a shift, and therefore cannot be handled as the character strings on the same line.

Many contents are created so that the user presses a link in order to change the screen. Therefore, at present, many links are arranged in the content. Therefore, it is necessary to recognize the user's link by reading, and at the same time notify the user that the link has been pressed correctly.
That is, the link location cannot be clearly notified by voice, and it is difficult to easily recognize that the transition from the link is in progress.

  In addition, some browsers have been modified and a reading interface has been added to realize reading aloud, but even in this case, general sites (such as HTML) can be displayed. However, the current situation is that it only supports specific sites.

  The object of the present invention is not only to be able to realize smooth speech reading on the screen text displayed on the browser, but also to clearly read out the link part by voice or to read out that the transition from the link is in progress. Accordingly, it is an object of the present invention to provide a character information display device with a speech synthesis function that can easily recognize the state of a browser and a control method thereof.

  In order to achieve the above object, a character information display device according to a first aspect of the present invention stores a character information including a display object including a character string and a display rule defining a display method of the display object. A display unit for displaying a display object stored in the storage unit, a speech synthesis unit for converting a character string into speech, and a character string included in the character information being displayed on the display unit as the speech synthesis unit In the case of conversion to voice, the display regulation of the character string to be converted to speech is referred to. If the predetermined display regulation is used, the first voice is converted to voice. If the predetermined display regulation is not used, the second is converted. And a controller that controls the speech synthesizer so as to convert the voice into voice.

  Preferably, the communication device further includes a communication unit that connects to a network and acquires the character information from a server connected to the network.

  Preferably, an operation unit for selecting and instructing at least one display object included in the character information displayed on the display unit is further provided, and the predetermined display rule is selected and instructed by the operation unit. It includes a selection position display rule indicating that

  Preferably, the predetermined display rule includes a link destination display rule indicating that a link destination is associated with a display object.

  Preferably, an operation unit for instructing or determining at least one display object included in the character information displayed on the display unit is further provided, and the control unit includes a display object associated with the link destination. When a selection instruction or a determination instruction is given by the operation unit, a character string included in the display object is converted into a voice by the voice synthesis unit using a third voice.

  Preferably, an operation unit for determining and instructing at least one display object included in the character information displayed on the display unit is further provided, and the control unit is configured such that the display object associated with the link destination is the operation object. When the determination is instructed by the unit, the character string included in the display object instructed to be determined is converted into speech by the speech synthesis unit after the communication unit accesses the link destination.

  Preferably, an operation unit that performs an instruction to select and determine at least one display object included in the character information displayed on the display unit, and the display rule is selected by the operation unit. The control unit displays a character string included in the display object defined by the selection position display rule when a determination instruction is given by the operation unit. Voice conversion is performed by the voice converter.

  According to a second aspect of the present invention, speech synthesis is performed in a character information display device that stores character information including a display object that includes a character string and a display rule that defines a display method of the display object, and displays the display object. A method of synthesizing a character string included in the display object into speech; and referring to a display rule for a character string to be converted into speech when the speech synthesis step converts the character string into speech. The character string of the display object specified by the predetermined display rule is voice-converted by the first voice, and the character string of the display object not specified by the predetermined display rule is voiced by the second voice. Converting.

  According to a third aspect of the present invention, speech synthesis is performed in a character information display device that stores character information including a display object that includes a character string and a display rule that defines a display method of the display object, and displays the display object. A speech synthesis program that can be executed by a computer, a speech synthesis step for converting a character string included in the display object into speech, and a character string that is converted into speech when the speech is synthesized in the speech synthesis step The step of referring to the display rule, the step of converting the character string of the display object defined by the predetermined display rule into a first voice, and the character string of the display object not defined by the predetermined display rule Comprises the step of performing voice conversion with the second voice.

  According to the present invention, a link location on a Web screen can be reproduced by voice, and the state of the browser screen can be easily recognized by voice.

1 is a block diagram illustrating an example of a system configuration of a mobile phone. It is the figure which illustrated an example of the external appearance of a mobile telephone, (a) is a front external view, (b) is a back external view, (c) is a front external view, (d) is a back external view. Each figure is shown. It is a flowchart for demonstrating the display of the information at the time of the browser starting based on this embodiment, and the voice reading-out operation | movement. It is a figure which shows the image of the display image in the specific style which concerns on this embodiment. It is a figure which shows an example of the correction value of the notified information which concerns on this embodiment, the current character size, and a style (link). It is a figure which shows the example of a storage in the storage area | region of the accumulation | storage management information before sorting of the character string which concerns on this embodiment, and a wording. It is a figure which shows the example of a storage in the storage area | region of the accumulation | storage management information after a character string sorting which concerns on this embodiment, and a wording. It is a figure which shows the example of an image of the reading-out request | requirement which concerns on this embodiment. It is a figure which shows the outline | summary at the time of displaying a Web description language. It is a conceptual diagram which shows the process outline | summary of the Web reading-out function which concerns on this embodiment. It is a figure for demonstrating that a voice can be reversed and read out also by an <addr> tag. It is a figure for demonstrating the sorting process of a X, Y coordinate. It is a figure for demonstrating the drawing request | requirement in case the cursor is over the link of a text. It is a figure for demonstrating a sorting algorithm. It is a figure which shows the basic sequence at the time of reading out the whole page. It is a figure which shows the reading-out sequence at the time of a line direction scroll.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram showing an example of a system configuration of a mobile phone 10 as a character information display device with a speech synthesis function of the present invention. FIG. 2 is a diagram illustrating an example of the appearance of the mobile phone 1.
The mobile phone 10 is a so-called foldable mobile phone having a movable mechanism. FIG. 2A is a front view in an opened state (open state), and FIG. 2B is a closed state (closed state). (C) is a side view in the open state, and (d) is a side view in the closed state.

The mobile phone 10 according to the present embodiment is configured to be able to display Web information (acquired information) acquired from the server 30 connected to the wireless communication network 20 on a display unit.
Further, the mobile phone 10 according to the present embodiment has a voice reading function in addition to the function of a normal mobile phone. For example, a character string notified as a drawing request from a browser is handled as reading character string information. It is configured to be able to display the same as a normal browser without any changes.

Furthermore, the mobile phone 10 according to the present embodiment is configured to include the following processing functions.
Based on the acquired Web information, the mobile phone 10 displays a display object to be displayed such as a character string, a pictorial symbol, and an image, a style defined by content operated on the server 30 that provides the display object, and the like The display rule is extracted, the display object and the display rule are associated with each other and stored in the storage unit, and the display object is displayed according to the extracted display rule such as the style. The display rules include display coordinates (X, Y), a font type such as Gothic font, a display format (style) for instructing additional display such as an underline, or a display size.
Then, when the activation of the speech reading function (speech synthesizer) is requested for reading in the display state of the acquired Web information, the mobile phone 10 has a style that defines the display method stored in the storage unit, etc. It has a function of converting a character string extracted from a display object with reference to a display rule into speech by a speech synthesizer.

When the mobile phone 10 converts the character string included in the character information being displayed on the display unit to speech, the cellular phone 10 refers to the display rule of the character string to be converted to speech. It has a function of voice-converting with voice and voice-converting with the second voice if it is not a predetermined display rule.
Here, the predetermined display rule includes a selection position display rule that indicates that the display object is instructed to be selected by the operation unit (that is, the cursor is applied). The predetermined display rule includes a link destination display rule indicating that a link destination is associated with a display object.
Then, when the display object associated with the link destination is instructed to be selected or determined by the operation unit, the mobile phone 10 has a function of converting a character string included in the display object into a third voice. .
In addition, when the display unit associated with the link destination is instructed to be determined by the operation unit, the mobile phone 10 may include a character string included in the display object instructed to be determined after the communication unit has accessed the link destination. Has a function to convert the sound.
Further, the mobile phone 10 has a function of converting a character string included in a display object defined by the selection position display rule into a voice when a determination instruction is given by the operation unit.

Further, when the cellular phone 10 requests the activation of the speech reading function (speech synthesizer) in the display state of the acquired Web information, the cellular phone 10 sorts the display objects stored in the storage unit with respect to each display coordinate and It has the function to convert to.
The mobile phone 10 has a function of storing correction values for display coordinates for each of a plurality of display formats, and sorting display objects after correcting with correction values according to the display format for each display object.
The mobile phone 10 has a function of storing correction values for display coordinates for each of a plurality of display sizes, and sorting display objects after correcting with correction values according to the display size for each display object.
When the activation of the speech reading function (speech synthesizer) is requested in the display state of the acquired Web information, the mobile phone 10 performs display at the cursor position from among a plurality of display objects stored in the storage unit. A display object associated with the display format is searched, and a character string included in the searched display object is converted into speech.

  Hereinafter, the configuration and function of each unit of the mobile phone 10 according to the present embodiment, and the voice reading control will be described in order.

As shown in FIG. 1, the mobile phone 10 includes a communication processing unit 11 including a transmission / reception antenna 111, a memory 12, a key operation unit 13, a dial input unit 14, a sub display unit 15, and a main display unit 16. A speech synthesis processing unit 17 including a speaker 171 and a microphone 172, a reading key input unit 18, and a control unit (CPU) 19.
2A, the main body case 100 of the mobile phone 10 includes a key input side main body case 101 as a first case and a display side main case 102 as a second case. These are connected by a movable mechanism (not shown) to form an open / close state.

The communication processing unit 11 performs a wireless communication operation via the base station, for example, a telephone number calling operation or an e-mail transmission / reception operation.
The communication processing unit 11 includes a transmission / reception antenna 111, and modulates voice information, e-mail, etc. processed by the control unit 19 to perform wireless communication using radio waves, and a base (not shown) by the transmission / reception antenna 111. The data is transmitted to the server 30 via the station and the communication network 20.
Further, the communication processing unit 11 demodulates various information such as electronic mail and voice information transmitted from the base station by radio and received by the transmission / reception antenna 111 and outputs the demodulated information to the control unit 19.
The communication processing unit 11 outputs Web information (acquired information) acquired from the server 30 connected to the wireless communication network 20 to the control unit 19.
In the present embodiment, the transmission / reception antenna 111 is built in the key input side main body case 101 or the display side main body case 102.

The memory (storage unit) 12 includes a nonvolatile memory such as an EEPROM, and stores a control program for sending and receiving calls and mails, an Internet browser, message data, an address book in which names and telephone numbers are registered, and the like. To do.
The memory 12 stores a speech reading database including a text string necessary for a reading function to be described later. In this database, text strings for reading are systematically arranged so that the connection relationship before and after is established as a sentence.
The memory 12 stores a control table and a weighting table for the voice reading function.
The memory 12 stores “standard text”, “abbreviated text”, and “explanatory text” for each menu item displayed on the display unit.
The memory 12 associates a display object extracted from the Web information by the control unit 19 with a display rule that defines a display method on the display units 16 and 15 that is defined by a server that provides the display object. Stored in a separate form. As described above, in this display rule, the selection position display rule indicating that the display object is selected and instructed by the key operation unit 13, and the link destination display rule indicating that the link destination is associated with the display object. including.
Further, in the memory 12, correction values for display coordinates are stored for each of a plurality of display formats by the control unit 19.
In the memory 12, the control unit 19 stores correction values for display coordinates for each of a plurality of display sizes.

The key operation unit 13 has an end (end call) / power key, a start (call) key, a plurality of numeric keys corresponding to numbers, etc., and when these keys are operated by the user, Input information is output to the control unit 19.
Further, whether or not to read out each item of the control table of the voice reading function stored in the memory 12 (ON / OFF) can be set through the control unit 19 by operating the key operation unit 13.
By operating the key operation unit 13, the user can issue an instruction to select and determine a display object included in the character information displayed on the display units 16 and 15.

As shown in FIG. 2C, the dial input unit 14 is disposed on the side surface of the display-side main body case 102 so that the user can easily operate with the thumb of the user when holding the mobile phone 10 in the open state. The dial-type input unit is configured to be operated in two directions.
By operating the dial input unit 14, the user can change the sound output volume and the character size displayed on the sub display unit 15 and the main display unit 16.
Further, as apparent from FIGS. 2C and 2D, when the dial input unit 14 in the closed state and the open state are compared, the operation directions in the upper and lower directions are physically reversed. In the form, in order not to cause a sense of incongruity for the user, the operation direction viewed from the user and the action on the operation (for example, the above-described volume change and display font size (character size to be displayed) change) always match It is controlled by the control unit 19.

The sub display unit 15 has a display device such as a liquid crystal display (LCD) and is visually recognized by the user in the closed state as shown in FIG.
The main display unit 16 includes a display device such as a liquid crystal display (LCD) and is visually recognized by the user in the open state as shown in FIG.
The sub display unit 15 and the main display unit 16 display the text of the received e-mail and various text data stored in the memory 12 under the control of the control unit 19 in the closed state and the open state, respectively.
In addition, the sub display unit 15 and the main display unit 16 are in the closed state and the open state, respectively, under the control of the control unit 19, the acquired Web information is stored in the display rule (display coordinates, display format, or / And display size).

The voice synthesis processing unit 17 has a voice processing circuit, and is connected to a speaker 171 that outputs voice for a call function and a microphone 172 that inputs voice.
The voice synthesis processing unit 17 performs predetermined processing on the voice collected by the microphone 172 and supplies the processed voice to the control unit 19. In addition, the voice synthesis processing unit 17 performs predetermined processing on the voice information supplied from the control unit 19 and outputs the voice information from the speaker 171.
Further, as shown in FIGS. 2A and 2B, the speaker 171 includes two voice output units, that is, a reception speaker 171a and a ringer speaker 171b, and outputs a voice as a processing result of the reading function.
Furthermore, the speech synthesis processing unit 17 has a speech synthesis circuit as a reading engine, and converts text data read out from the memory 12 and extracted by the control unit 19 into speech data when performing speech reading, The voice is synthesized by the receiving speaker 171a or the ringer speaker 171b, which is a unit, and output as a voice.
When the voice synthesis processing unit 17 performs voice conversion under the control of the control unit 19, for example, if there is a link destination and a predetermined display rule such as cursor position display, the voice synthesis processing unit 17 performs voice conversion with the first voice, If the display is not stipulated, voice conversion is performed with the second voice. When the display object associated with the link destination is instructed to be selected or determined by the key operation unit 13, the character string included in the display object is changed to the first voice. Voice conversion is performed with the voice of 3.

As shown in FIG. 2B, the reading key operation unit 18 includes a push-down button 18a disposed in the center of the display-side main body case 102, and an input circuit for switch input by this button. The
The cellular phone 10 according to the present embodiment has a reading function. When the push-down button 18a is pressed (operated), for example, the voice is output from the ringer speaker 171a in the closed state, and the voice is output from the receiving speaker 171a in the open state. In this manner, the control unit 19 controls the operation.

  The control unit 19 is mainly composed of a microcomputer and controls the entire mobile phone 1. For example, the control unit 19 controls the wireless transmission / reception of various types of information in the communication processing unit 11, the processing of audio information for the audio processing unit 17, the display control of information on the main display unit 16, and the input information of the key input unit 13. Responsive processing, access control to the memory 12, and the like are performed.

  Basically, when the user operates the push button 18a, the control unit 19 executes a function for reading out the displayed text. At that time, as a reading function, a text reading method is used in which a text string is extracted / generated and the text is read.

As will be described in detail later, the control unit 19 starts the browser, extracts the display object and the display rule defined for each content on the server 30 that provides the display object from the acquired Web information, and displays the display rule. The object and the display rule are associated with each other and stored in the memory 12, and the display object is displayed on the main display unit 16 or the sub display unit 15 according to the extracted display rule.
When the control unit 19 displays the acquired Web information on the main display unit 16 or the sub display unit 15, for example, when the voice synthesis processing unit 17 is requested to be activated by the operation of the reading key operation unit 18, A character string extracted from the display object with reference to the display rule stored in the memory 12 is converted into speech by the speech synthesis processing unit 17.

When the character string included in the character information being displayed on the display units 16 and 15 is converted into speech, the control unit 19 refers to the display rule of the character string to be converted into speech, and has a predetermined display rule (there is a link destination) Voice synthesis processing so that the voice is converted with the first voice in the case of the cursor position display, etc., and the voice is converted with the second voice whose sound quality is different from the first voice when the display is not prescribed. The unit 17 is controlled.
In addition, when the display operation associated with the link destination is instructed to be selected or determined by the key operation unit 13, the control unit 19 converts the character string included in the display object into a third sound. The voice synthesis processing unit 17 is controlled.
As described above, the control unit 19 according to the present embodiment changes the voice quality, speed, intonation, and the like of reading according to the drawing style, and changes the voice quality, speed, and intonation of reading when the selectable object changes. Has a function of controlling the speech synthesis processing unit 17.
When the display object associated with the link destination is instructed to be determined by the key operation unit 13, the control unit 19 accesses the link destination by the communication unit and then includes the character string included in the display object instructed to be determined. The voice synthesis processing unit 17 is controlled so as to convert the voice into a voice. Further, when a determination instruction is given by the key operation unit 13, the control unit 19 controls the speech synthesis processing unit 17 so as to convert the character string included in the display object specified by the selection position display rule into speech.

Note that the control unit 19 is requested to activate the speech synthesis processing unit 17 by, for example, operating the reading key operation unit 18 in a display state in which the acquired Web information is displayed on the main display unit 16 or the sub display unit 15. Then, the display objects stored in the memory 12 are sorted with respect to the respective display coordinates, and then the voice synthesis processing unit 17 performs conversion into voice.
The control unit 19 stores correction values for the display coordinates for each of the plurality of display formats in the memory 12. Then, the control unit 19 sorts the display objects after correcting with the correction values stored in the memory 12 according to the display format for each display object. The control unit 19 stores correction values for display coordinates for each of the plurality of display sizes in the memory 12. Then, the control unit 19 sorts the display objects after correcting with the correction value according to the display size for each display object.

  In addition, the control unit 19 is requested to start the speech synthesis processing unit 17 by operating, for example, the reading key operation unit 18 in a display state in which the acquired Web information is displayed on the main display unit 16 or the sub display unit 15. A display object associated with a display format for performing display at which the cursor is located among a plurality of display objects stored in the memory 12, and a speech synthesis processing unit for a character string of the searched display object 17 is converted to voice.

  The control unit 19 interrupts reading when another screen is displayed, and controls so that even if blinking or the like is designated, even if a plurality of drawing requests are notified for the same character string, the reading is performed only once. The control unit 19 controls the speech synthesis processing unit 17 so as to read out the character strings that are divided and notified in some cases when reading out with the same voice quality. The control unit 19 buffers the newly displayed character string during reading in order to prevent reading interruption. Further, the control unit 19 interrupts the reading when another screen is displayed, and controls the speech synthesis processing unit 17 so that the reading is interrupted and the corresponding object is read when the cursor transitions to a selectable object. To do. Further, the control unit 19 determines the range to be read out by the coordinate value for a character string partially protruding from the display areas of the display units 16 and 15 in order to prevent overlapping reading. The control unit 19 is configured to notify the character string again by a redrawing request when the character string is not notified, such as a cache display.

  Next, the operation according to the above configuration will be described with reference to FIG. 3 to FIG. 8 with a focus on information display and speech reading operation when the browser is activated.

FIG. 3 is a flowchart for explaining the display of information and the speech-to-speech operation when the browser is activated.
FIG. 4 is a diagram illustrating an image of a display image in a specific style.
FIG. 5 is a diagram showing an example of the notified information, the current character size, and the correction value of the style (link).
FIG. 6 is a diagram illustrating an example of storage in the storage area for storage management information and wordings before sorting character strings.
FIG. 7 is a diagram showing an example of storage in the storage area for storage management information and wordings after character string sorting.
FIG. 8 is a diagram illustrating an image example of a reading request.

When the browser is activated (ST1) and a drawing start request notification is issued (ST2), the character string, style, and coordinates to be drawn are notified (ST3).
Next, it is determined whether or not style information is being selected from the acquired information (ST4).
If it is determined in step ST4 that it is not currently selected, the acquired character string is stored (buffered) in, for example, the memory 12 (ST5).
Next, it is determined whether or not the acquired style is a correction target style (ST6).
If it is determined in step ST6 that the style is a correction target style, the coordinate value is corrected (ST7) and the process proceeds to step ST8. If it is determined that the style is not a correction target style, step ST8 is performed without performing the correction process in step ST7. Move on to processing.
In step ST8, it is determined whether or not the coordinates are a drawing screen.
If it is not a drawing screen, the character string is discarded (ST9), and the process proceeds to step ST10. If it is a drawing screen, the process proceeds to step ST10 without passing through step ST9.
In step ST10, it is determined whether or not the drawing process has ended. If it has not ended, the process proceeds to step ST2.
If it is determined in step ST10 that drawing has ended, character string sorting processing is performed (ST11), and character strings of the same style are passed (ST12).
If it is determined in step ST4 that the style is being selected, the object is read aloud (ST13), and the character string buffer is cleared (ST14).

In the present embodiment, a character string notified as a drawing request from the browser is handled as read-out character string information.
In each main step, the following processing is specifically performed.

The coordinate correction in step ST7 is as follows.
For example, as shown in FIG. 4, since the coordinate position is shifted in drawing in a specific style, the coordinate position is corrected according to the display format (style) and the character size.
Correct the coordinate position of a special display object (link) such as "Apple". When the link style is notified by the drawing request, a correction value corresponding to the current character size is determined and corrected from the database for coordinate correction.
For example, as shown in FIG. 5, the notified “apple” information has coordinate values X = 0, Y = 5, style “link”, and the number of characters “3”, and the current character size setting value Is “character size standard”, and the correction value of the style (link) is an example in which the small character size is “Y-3”, the standard character size is “Y-5”, and the large character size is “Y-8”. Then, the coordinate position is corrected as follows.
Based on the above information, the coordinate value is corrected. Since the character size is standard in the style (link), the Y coordinate of the three characters “apple” is added by −5, and the coordinate value is (X: 0, Y: 0).

In step ST11, if the drawing request is read out in the notified order, it may not be a sentence. Therefore, the sorting process is performed using the coordinate value attached to the character string.
The coordinate value is a value after correction processing.
FIG. 6 shows an example of storage in the storage area of the storage management information and wordings before sorting of character strings, and FIG. 7 shows an example of storage after character string sorting.
In this example, the words before the character string sorting are “fruit:”, “100 yen”, “two”, “mandarin orange”, “200 yen”, “melon”, “300” as shown in FIG. As shown in FIG. 7, the order of “yen”, “strawberry”, “400 yen”, and “apple” is “fruit:”, “apple”, “100 yen”, “ The order is “2”, “mandarin orange”, “200 yen”, “melon”, “300 yen”, “strawberry”, “400 yen”.

Further, since a different drawing style is notified for each display object, reading is performed according to the display object.
Taking the screen image of FIG. 4 as an example, the voice of the link character string different from the standard (set voice) is read out.

Also, the object to which the cursor is transitioned is specified by the drawing style, and the corresponding character string is read out by changing the type of sound.
Taking the screen image of FIG. 4 as an example, the character string “apple” is read out in a voice different from the standard.

In addition, since a drawing request is notified for each row or object, smooth reading is performed by buffering and reporting a plurality of drawing requests together to the reading engine (control unit and speech synthesis processing unit).
For example, as shown in FIG. 8, even if a character string is notified line by line, it can be read out by the same reading method ignoring a line feed or the like.

  In addition, line scrolling during reading is performed by buffering a newly displayed line and passing it to the reading engine when reading is completed.

  For page scrolling and jumping to another screen, the character string being read out is discarded and reading is performed from the top of the page.

A character string notified between a drawing start request and a drawing end request is set as a reading target.
In addition, the character string notified in the same coordinates is the character string notified in advance.

  When the cursor is moved to a selectable object, the character string being read out is interrupted and the corresponding object is read out.

  In some cases, the character string may be missing from the upper or lower part of the screen display. In this case, the range to be read out is determined by the coordinate value.

  When displaying a screen or the like stored in the cache, a drawing request is not notified, and a character string is acquired by performing a redrawing request.

An object having no character string is determined by style, and a specific wording is read out.
For example, for an object that does not have a character string such as a radio button, when the transition or determination is made, the character string in the processing unit is transferred to the control unit that is an engine, so that reading is realized.

  Next, a specific process including a browser function for content including a link will be described.

  In the mobile phone 10 of the present embodiment, the following processing is performed on the content including the link.

1. When the cursor transitions to the link, reading is performed with a sound quality different from the set value (for example, a female voice when the set sound quality is male voice).
This is determined by the character type (link character) notified from the browser.
When the browser recognizes the link, it draws the link with conditions such as link characters (italics, blue, underline).

2. The character string of the link is handled as the title of the next screen.
When the cursor transitions to the link, all the character strings specified for the link are acquired, and after the transition from the link to the next screen, the character string of the link is read out, thereby producing a title reading effect.
The transition by pressing the link determines the link destination when the link transition destination information (URLto) is notified.

3. Notify the user that the link is in progress.
When transitioning to the next screen by pressing the link, the character string indicating that the next screen transition is being stored in the memory 12 is read out until the next screen starts drawing, and the user is linking to the next screen. Notify you by reading aloud.

4). Read all strings specified in the link.
Even when the screen is changed by pressing the link while reading the link character string, it is possible to read the entire link character string.

5. Only the character string of the link in the screen is extracted, and only the link can be read out. When the cursor transitions to the link, it is read out with the set sound quality (for example, male voice), and when the link is pressed, it is read out with a sound quality different from the set value (for example, when the set sound quality is male voice).

6). After the link is pressed, if the notification of screen transition has not been notified to the terminal side for a certain period of time, a character string indicating that the communication is ongoing is stored in the memory separately indicating that the communication is ongoing to the user. Notify by reading aloud.

A description example (link part) of content including a link is as follows.
<a href = http: // kyocera. jp title = Kyocera> Jump </a>

The contents of the Web description language are as follows, and an outline thereof is shown in FIG.
1. The link destination is described in the link designation tag (a tag).
2. Title is displayed in the soft key area (displayed in the guide).
3. The “jump” part is the actual link string.

The browser analyzes the Web description language and notifies the terminal side of the following information.
[When link screen is displayed]
Write “Jump” to the coordinates X and Y in the link character style.
[When the cursor moves to a link]
Invert the colors of coordinates X and Y.
Write “Kyocera” on the guide.
[When link is pressed]
Since the URL is notified, transition.

When a screen including a link is displayed, notification is performed by determining the character size and character type for the character string from the Web description language acquired by the browser.
When the browser acquires “jump” of the character string of the link part, the display setting for the link is made as “blue characters, italic characters are displayed underlined at coordinates X and Y”.
When the reading key 18 is pressed, reading is performed from the beginning to the end of the screen. Character string drawing information used for screen drawing is stored in a memory, and the coordinates of the character string are sorted and read out in order from the top of the coordinates (upper part of the screen). The link character “jump” is specified as “blue character, italic character underlined and displayed at coordinates X, Y”. Change the voice type and read aloud.
If the link is displayed continuously, the voice type is continuously set to read out, but if the normal text is displayed from the link character string, the voice type is restored.

When the cursor is moved by the user's key operation and the cursor transitions to the link, the coordinates where “jump” of the link character string is designated are specified as a range, and the color is reversed and displayed. The browser recognizes that the title “Kyocera” is set in the link part when the cursor transitions to the link, and displays “Kyocera” in the character key guide part on the display part.
The voice is treated as if the cursor has transitioned to the link based on the reversed opportunity, and the voice type is changed and read out. After drawing the entire screen, the command “invert the color of a specific coordinate” is only when the cursor transitions to the link, so the character string being read out is discarded and the cursor transitions preferentially and the link character string is read out.

As shown in FIG. 9, the mobile phone 10 includes soft keys SFTK <b> 1 to 3. The soft key here is a key that is assigned a function that is frequently used every time the screen state changes, and has a guide area at the bottom of the display unit 16, In this guide area, the function currently assigned to the soft key is displayed according to the screen. In many cases, a plurality of soft keys are provided including the decision key SFTK2.
When the user operates the direction key, the control unit 19 moves the cursor on the screen of the display unit 16 according to the pressed direction. At this time, when the cursor is placed on the “jump” which is the link character string, the display color inversion processing as described above is performed. That is, the browser function of the control unit 19 updates the drawing after adding the display rule of color inversion processing as the drawing of the display unit 16 to the display rule acquired as Web information. When a color reversal process is added to this display rule, a character string “title Kyocera” is also acquired and displayed in a location corresponding to the determination key SFTK2 in the guide area.

When the link is pressed, the link destination URL “http://kyocera.jp” is set as the transition destination to communicate with the server 30, and the transition destination based on the new Web description language acquired from the server 30 is used. Draw the screen.
When a communication is started, the timer counts, and if a new screen is not drawn after a certain period of time, "Ready to connect" can be read out.
When the link is pressed, the title “Kyocera” of the link source is stored in the memory, and when drawing of the new screen is completed, the title effect is produced by reading “Kyocera” before reading the character string of the new screen. .

FIG. 10 is a conceptual diagram showing an outline of processing of the Web reading function according to the present embodiment.
This reading-out function is all processed under the control unit of the control unit 19, the device layer 191 includes a browser 192, and further includes a buffer 193, a speech interface 194, and a speech engine unit 195. The speech engine unit 195 includes the function of the speech synthesis processing unit 17.

The processing of the Web reading function is performed as follows.
1. HTML (reading target) is acquired from the server 30.
2. The browser 192 requests the device layer 191 to draw characters in the HTML. The device layer 191 stores this in the buffer 193.
3. The browser 192 notifies the device layer 191 of drawing completion.
In response to this request, a request is made to read out the words stored in the speech interface 194.
4). The speech engine unit 195 converts the character string into speech and reads it out.

In the present embodiment, the voice can be inverted and read out even with an <addr> tag indicating that address information as shown in FIGS. 11A and 11B is linked.
The title example in this case is as follows.
<addr title = “addr link click”> html </ addr>
Thus, there are many examples of display rules. Of course, if there is a link with a telephone number associated with a mail number or a mail address, a rule different from usual is added to the display rule.

The procedure is as follows from the time when the drawing character string is received from the browser 192 by the character string notification command to the time when the task of the speech engine unit 195 is performed as the reading character string.
1. Notification of screen update completion,
2. Sort by X and Y coordinates,
It is a procedure.
Hereinafter, each procedure will be described.

1. Screen update completion notification The browser 192 makes a drawing request many times in response to a character string notification command. Therefore, it is necessary to detect from the browser that the screen drawing is finished.
This is realized by detecting a notification function indicating that the WML format issued from the browser 192 has been changed, and detecting a notification function indicating that the subsequent screen update has been completed.

2. Sorting by X and Y coordinates A character string to be drawn is notified from the browser to the device layer by a character string notification command. As shown in FIG. 12, a character string drawn in order from the top is notified with the upper left corner excluding the pictogram area of the display unit 16 as the base (0, 0).
However, when a character string whose link destination is specified by text is included and the cursor is placed on the link, the character string notification command makes a rendering request only for that character string at the end.
The state is shown in FIGS. 13 (A) and 13 (B).

If the character string is read out in the order of the drawing request of the browser 192, “ABCDEFGHIJIKLPQRSTUVWXYZMNO” is obtained. In other words, in order to read it out in the correct order, it is necessary to sort the character strings. This is performed based on the (X, Y) coordinates of the character string notification command of the drawing request.
14A to 14C show the sorting algorithm.
In this example, sorting is performed in ascending order by Y coordinate. However, if the Y values are the same, the X coordinate values are compared in ascending order. If the value of X is also the same, the one with the drawing request first is given priority.
As a result of the sorting process, the read-out character string is “ABCDEFGHIJKLMNOPQRSTUVWXYZ”.

  Next, full screen drawing / scroll drawing determination in Web reading will be described.

<Full screen drawing judgment>
FIG. 15 is a diagram showing a basic sequence for reading the entire page.
Drawing from immediately after the drawing start command is called to before the drawing end command for drawing completion is called is regarded as full-screen drawing.
In this case, the character string, drawing coordinates, and link / normal with focus are stored in the buffer 193 and drawn.
When the device layer 191 is notified of the completion of drawing, it waits for a predetermined time, for example, 1 second, sorts by coordinates, and stores it in the sort buffer.
The device layer 191 designates voice. When normal, the voice is set, and when the link is focused, the voice is different (reverse) from the setting. For example, if the setting is male voice, female voice is set.
The device layer 191 requests the speech interface 194 to read out the words of the same voice type together. The device layer 191 repeats this request processing while there is a word in the sort buffer until a notification of completion of reading is received.

<Scroll drawing determination>
FIG. 16 is a diagram illustrating a reading sequence when scrolling in the line direction.
Drawing from the drawing after the scroll start command is called until the drawing end command is called is regarded as scroll drawing.
If the scroll start command is called with a plus (+) value, it is determined to scroll downward, and if called with a minus (-) value, it is determined to scroll upward.
In the case of drawing for moving a line that has already been drawn, it is not stored in the buffer, but in the case of drawing of a newly appearing line, it is stored in the buffer for reading out.
Then, the device layer 191 requests the speech interface 194 to collectively read out the words of the same voice type. The device layer 191 repeats this request processing while there is a word in the sort buffer until a notification of completion of reading is received.

According to the present embodiment, when a character string included in character information being displayed on the main display unit 16 is converted into speech, the display rule of the character string to be converted into speech is referred to and a predetermined display rule (link destination) If there is a cursor position display, etc., the voice is converted with the first voice, and if it is not a predetermined display rule, the voice synthesis is performed so that the voice is converted with the second voice whose sound quality is different from the first voice. When the processing unit 17 is controlled and the display object associated with the link destination is instructed to be selected or determined by the key operation unit 13, the character string included in the display object is converted into speech by the third voice. Since the control unit 19 that controls the speech synthesis processing unit 17 is provided as described above, it is possible to clearly display the link location by voice and easily recognize that the transition from the link is in progress.
In addition, the coordinate value is corrected according to the notified character string style, and reading is performed after sorting according to the coordinates instead of the notified order, and the voice quality, speed, intonation, etc. are determined according to the drawing style. Since the voice quality, speed, and intonation of the reading are changed at the transition of the selectable object, and the same character string such as blinking is notified, it is configured to read out only once. Can be obtained.

Smooth voice reading can be realized.
Since it is read out using a drawing request, it can be realized without modifying the browser, and as a result, a display equivalent to that of a normal browser is possible.
In the case of reading with the same voice quality, it is possible to prevent the reading from being interrupted and to increase the probability that the idiom is read correctly by reading the character strings that are divided and notified in batches.
Further, during reading, since the newly displayed character string is buffered, the buffered character string can be read out after the reading is completed, thereby preventing the reading from being interrupted.
Further, when another screen is displayed, the reading can be interrupted, and the screen and the reading can be synchronized.
Further, when the cursor transitions to a selectable object, the reading can be interrupted and the corresponding object can be read out, so that the reading can be performed without missing the selected timing.
Further, for character strings partially protruding from the display area, the range to be read out can be determined by the coordinate value, and overlapping reading can be prevented.
If the character string is not notified, such as a cache display, the character string can be notified again by a redraw request, and the same screen is displayed even if the character string is obtained and redrawn, so flicker does not occur .
Further, by determining an object having no character string by style, a specific character string can be given and read out.

  The speech reading process as described above is recorded as a voice reading program on a recording medium, a semiconductor storage device (memory), an optical disk, a hard disk or the like that can be read by a terminal device (computer), and read by the terminal device. Executed.

DESCRIPTION OF SYMBOLS 10 ... Mobile phone 100 ... Main body case 101 ... Key input side main body case 102 ... Display side main body case 11 ... Communication processing part 111 ... Transmission / reception antenna 12 ... Memory 13 ... Key operation part 14 ... Dial input part 15 ... Sub display part 16 ... Main display unit 17 ... Speech synthesis processing unit 171 ... Speaker 172 ... Microphone 18 ... Reading key operation unit 19 ... Control unit 191 ... Device layer 192 ... Browser 193 ... Buffer 194 ... Speech interface 195 ... Speech engine unit 20 ... Wireless communication network 30 …server

Claims (8)

A storage unit that stores character information including a display object that includes a character string and a display rule that defines a display method of the display object;
A display unit for displaying a display object stored in the storage unit;
A speech synthesizer that converts text into speech;
When the character string included in the character information being displayed on the display unit is converted to speech by the speech synthesizer, the character string to be converted to speech is a display object associated with a link destination. A controller that controls the speech synthesizer to perform speech conversion with the first speech and convert the speech with the second speech if the display object is not associated with the link destination;
An operation unit;
With
When the display unit associated with the link destination is selected or determined by the operation unit, the control unit transmits the character string included in the display object associated with the link destination with a third voice. Character information display device with speech synthesis function for voice conversion. The character information display device with a speech synthesis function according to claim 1, further comprising a communication unit that connects to a network and acquires the character information from a server connected to the network. When the display unit associated with the link destination is instructed to be determined by the operation unit, the control unit is included in the display object instructed to determine after the communication unit has accessed the link destination. The character information display device with a speech synthesis function according to claim 2, wherein the speech synthesis unit converts the character string to be speech-converted. A storage unit that stores character information including a display object that includes a character string, a display rule that defines a display method of the display object, a display unit that displays a display object stored in the storage unit, and a character string A voice synthesizing unit for converting to speech, and a character string to be converted to speech when the character string included in the character information being displayed on the display unit is converted to speech by the speech synthesizing unit The voice synthesizer is controlled so that voice conversion is performed with the first voice in the case of the attached display object, and voice conversion is performed with the second voice when the display object is not associated with the link destination. In a control method of a character information display device with a speech synthesis function comprising a control unit and an operation unit,
When the display object associated with the link destination is instructed to be selected or determined by the operation unit, a step of voice-converting a character string included in the display object associated with the link destination with a third voice is performed. A method for controlling a character information display device with a speech synthesis function. The control method of the character information display device with a voice synthesis function according to claim 4, further comprising a communication unit that connects to a network and acquires the character information from a server connected to the network. When the display unit associated with the link destination is instructed to be determined by the operation unit, the control unit is included in the display object instructed to determine after the communication unit has accessed the link destination. The method for controlling a character information display device with a voice synthesis function according to claim 5, wherein the voice synthesizer converts the character string to be voiced into a voice. A display unit, a speech synthesis unit for converting a character string into speech,
When the character string included in the character information being displayed on the display unit is a display object associated with a link destination, the first sound is used, and when the character string is not a display object associated with a link destination, the first sound is used. A control unit that controls the voice synthesis unit that performs voice conversion using two voices;
An operation unit,
When the display unit associated with the link destination is selected or determined by the operation unit, the control unit transmits the character string included in the display object associated with the link destination with a third voice. Character information display device with speech synthesis function for voice conversion. A display unit, a speech synthesis unit for converting a character string into speech,
When the character string included in the character information being displayed on the display unit is a display object associated with a link destination, the first sound is used, and when the character string is not a display object associated with a link destination, the first sound is used. A control unit that controls the voice synthesis unit that performs voice conversion using two voices;
A control method of a character information display device with a speech synthesis function comprising an operation unit,
When the display unit associated with the link destination is selected or determined by the operation unit, the control unit transmits the character string included in the display object associated with the link destination with a third voice. A method for controlling a character information display device with a voice synthesis function for voice conversion.

Images