JP2000267837A - Man-machine interface device and recording medium recording man-machine interface control program - Google Patents

Abstract

(57) [Summary] [Problem] When controlling a device having a graphical user interface (GUI) using voice recognition, it is difficult for an operator to understand a keyword to be uttered. Solve. SOLUTION: While turning on a voice input switch 8 for voice recognition, measures are taken to make display characters easier to read, such as lowering the brightness of the whole or a part of the screen, and objects selectable on a GUI screen A keyword to be uttered for speech recognition or a serial number for identification is displayed on or near the upper side by letters to assist the operator so that smooth operation can be performed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the technical field of a man-machine interface, and more particularly, to a man-machine interface that enables a GUI (graphical user interface) on a screen to be operated by a human voice on a computer or a television. The present invention relates to a machine interface device and a recording medium recording a man-machine interface control program.

[0002]

2. Description of the Related Art Conventionally, a selection operation using a human voice on a GUI screen employs a method of uttering a character displayed along with an icon or the like. In general, a method of uttering and selecting a keyword stored in advance or a method of uttering and selecting a target item from a keyword list displayed in advance on a screen as a list is generally used. In addition, for example, a car navigation system, voice recognition software operating on a personal computer (personal computer), and the like have been developed to provide such a selection operation function.

[0003]

However, in the above-described conventional method, when an object whose keyword is not clearly displayed is selected (for example, HTML)
In the case of selecting an image on [hypertext markup language], for example, the character information is not displayed. Therefore, it is not possible to know the keyword to be uttered by voice operation, and to obtain operability equivalent to a pointing device such as a mouse. Therefore, it is necessary to display a keyword to be uttered in some way.

On the other hand, in the method of displaying a keyword list as a list in advance, the list is displayed even when it is unnecessary, which may be a restriction on screen design.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and it is an object of the present invention to allow an operator to clearly know keywords to be uttered and selectable items when operating a GUI screen by voice. And a recording medium recording a man-machine interface control program.

It is a further object of the present invention to provide a man-machine interface device in which a keyword is displayed only when an operator speaks, so that the keyword hardly affects the configuration of the GUI screen itself. An object of the present invention is to provide a recording medium recording a man-machine interface control program.

A further object of the present invention is to provide a man-machine interface device and a recording medium for recording a man-machine interface control program in which the correspondence between an object and a keyword is clear and the visibility of the keyword is improved. Is to provide.

[0008]

In order to achieve the above object, the invention of a man-machine interface device according to the first aspect of the present invention provides a GUI screen for creating and displaying a GUI (graphical user interface) screen having a voice recognition function. Display means, voice input means for inputting voice to the GUI screen creation / display means, and turn on the voice input
/ OFF switch, and a character string creation / display means for generating / displaying a character string for assisting the operator in conjunction with ON of the switch.

The character string creation / display means generates a character string for assisting the operator before the switch is turned on, and links the generated character string in conjunction with the turning on of the switch. It can be characterized by displaying.

The character string creating / displaying means includes means for extracting a keyword such as a file name or tag from an object such as an icon or image being displayed in response to the switch ON, and a means for creating / displaying the GUI screen. Means for controlling a display means to superimpose the keyword at a place where the object is located.

If the keyword cannot be extracted, the character string creating / displaying means generates a serial number for automatic identification, and controls the GUI screen creating / displaying means so that the position of the object is determined. A means for superimposing the serial number or the like at a location where the number is to be changed.

[0012] The character string creating / displaying means may include means for lowering the luminance of the entire screen or the luminance of the background of the character portion when performing the superimposing.

[0013] The keyword may be superimposed on or near the object.

According to a seventh aspect of the present invention, there is provided a recording medium having a GUI on a display screen by a computer.
(Graphical User Interface) is a recording medium recording a control program of a man-machine interface device provided with a GUI screen creation / display means capable of operating a human voice, wherein the control program is stored in a computer. On the other hand, a character string for assisting the operator is generated and displayed in conjunction with the ON of a switch for turning ON / OFF a voice input to the GUI screen creation / display means.

The recording medium according to the present invention has a GUI screen creating / display means for enabling a computer to operate a GUI (graphical user interface) on a display screen with human voice. A recording medium recording a control program of a man-machine interface device, wherein the control program causes a computer to generate a character string for assisting an operator,
A character string for assisting the generated operator is written in the GU.
Turn on / off voice input to I screen creation / display means
It is characterized in that it is displayed in conjunction with the ON of the F switch.

Here, the control program causes the computer to extract keywords such as file names and tags from objects such as icons and images being displayed in response to the switch ON, and causes the GUI screen creation / display means to The keyword may be superimposed at a location where the object is located.

Further, the control program causes the computer to generate a serial number or the like for automatic identification when the keyword cannot be extracted, and controls the GUI screen creation / display means to locate the object. Where the serial number or the like is superimposed at the place where the information is to be written.

The control program may cause the computer to reduce the luminance of the entire screen or the luminance of the background of the character portion when performing the superimposing.

(Operation) In the present invention, in order to clearly display a keyword to be uttered on the GUI screen, an ON / OFF switch of a microphone used when performing voice input is used, and the microphone is turned on. Sometimes display keywords on screen. As a display method, the whole GUI screen or
The brightness of a part near the object is reduced, and a character string such as a keyword is displayed there to clarify the comparison with the object, thereby ensuring the visibility of the keyword.

[0020]

Embodiments of the present invention will be described below in detail with reference to the drawings.

(First Embodiment) FIG. 1 shows a schematic configuration of a man-machine interface device according to an embodiment of the present invention. Here, 1 is an information source (communication path, storage device, etc.),
2 is a control device, 3 is a GUI screen generation device, 4 is a display device (CRT or the like), 5 is a voice recognition device, 6a and 6b are communication devices, 7 is a microphone, and 8 is a microphone switch. The communication devices 6a and 6b transmit audio signals and control signals from the switch 8, and may be wired or wireless (including infrared communication).

FIG. 2 shows an example of a display screen according to the present invention by the apparatus shown in FIG. 1. A selection screen as shown in FIG. 2A is displayed as described later. When the switch 8 is pressed, as shown in FIG. 2B, the entire screen is darkened and each keyword of the selection item is superimposed. FIG.
5 is a flowchart showing a control operation procedure of the control device 2 according to the present invention. Note that S1 to S11 represent each processing step.

Next, the operation of one embodiment of the present invention will be described with reference to FIGS.

Based on the data input from the information source 1 (S1), the control device 2 controls the GUI screen generating device 3 to display the GUI screen and the contents of the information source on the display device 4 (S2). . At this time, the control device 2 extracts in advance keywords that are features such as file names and tags from selectable objects such as icons and images (S3). If an appropriate character string cannot be obtained as a keyword (S4), a serial number or the like for identification is automatically generated (S5).

When using the voice recognition function, the operator speaks toward the microphone 7 while pressing the microphone switch 8. At this time, information on the microphone switch 8 is sent to the control device 2 and the GUI screen generation device 3 via the communication device 6a and the communication device 6b.

The control device 2 includes a microphone switch 8
In response to the input of the information (S6), as shown in FIG. 2, after taking measures to improve the visibility of the characters, such as lowering the luminance of the entire screen or the luminance of the background of the character portion, (S7) The CPU 3 controls the GUI screen generation device 3 to superimpose the keyword at the location (on or near the object) where the object is located (S7).
9). If the keyword has not been extracted (S8), the GUI screen generating device 3 is controlled to superimpose the serial number or the like at the location (on or near the object) where the object is located (S1).
0).

Further, the voice signal picked up by the microphone 7 is sent to the voice recognition device 5 via the communication devices 6a and 6b, and the voice recognition result in the voice recognition device 5 is sent to the control device 2, The control device 2 controls the entire system according to the content of the recognition result, and performs an operation such as an appropriate display (S11).

(Modification of First Embodiment) S3 in FIG.
Steps S4 and S5 may be between steps S1 and S2.

Further, as shown in S14, S16 and S18 in FIG. 4, the processing of the steps S3, S4 and S5 in FIG. 3 can be performed after the microphone switch 8 is turned on.

(Other Embodiments) The present invention relates to a plurality of devices (for example, a host computer, a microphone,
The present invention may be applied to a system including a modem device, a reader, a printer, and the like, or may be applied to a device including one device (for example, a next-generation television receiver, a car navigation device, and the like).

Another object of the present invention is to provide a recording medium (storage medium) recording software program codes for realizing the functions of the above-described embodiments to a system or an apparatus, and to provide a computer for the system or the apparatus. Needless to say, the present invention is also achieved when the CPU (or the CPU or the MPU) reads out and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the function of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.
As a recording medium for recording the program code and for recording variable data such as a table, for example, a floppy disk (FD), hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, non-volatile A memory card (IC memory card), a ROM, and the like can be used.

[0032]

As described above, according to the present invention,
When operating the GUI screen by voice, the operator can clearly know the keywords to be uttered and selectable items.

According to the present invention, the keyword is displayed only when the user speaks (when the microphone switch is ON), so that the configuration of the GUI screen itself is hardly affected.

Further, according to the present invention, since superimposition is performed on or near an object, the correspondence between the object and the keyword becomes clear.

Further, according to the present invention, since the luminance of the background of the keyword is reduced, the visibility of the keyword is also improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a man-machine interface device according to an embodiment of the present invention.

FIG. 2 is a view showing an example of a display screen according to the present invention by the apparatus of FIG. 1;

FIG. 3 is a flowchart showing a control operation procedure of the control device of FIG. 1 according to the present invention.

FIG. 4 is a flowchart showing another example of the procedure of the control operation according to the present invention of the control device of FIG. 1;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Information source (communication path / storage device etc.) 2 Control device 3 GUI screen generation device 4 Display device (CRT etc.) 5 Voice recognition device 6a, 6b Communication device 7 Microphone 8 Microphone switch

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 3/00 571K F-term (Reference) 5D015 KK01 LL00 5E501 AA01 AB13 BA09 CA03 CB15 EA21 EB05 FA05 FA42 FB29 FB44

Claims (11)

[Claims] 1. A GUI screen creation / display means for creating and displaying a GUI (Graphical User Interface) screen having a speech recognition function, and a speech input means for inputting a voice to the GUI screen creation / display means. A switch for turning on / off the voice input; and a character string creating / displaying means for generating / displaying a character string for assisting an operator in conjunction with the ON of the switch. Machine interface device. 2. The character string creation / display means generates a character string for assisting an operator before the switch is turned on, and converts the generated character string into an O of the switch.
2. The man-machine interface device according to claim 1, wherein the display is performed in conjunction with N. 3. The character string creation / display unit includes: an extraction unit that extracts a keyword such as a file name or a tag from an object such as an icon or image being displayed in response to the switch ON; 3. The man-machine interface device according to claim 1, further comprising: means for controlling a display means to superimpose the keyword at a position where the object is located. 4. If the keyword cannot be extracted, the character string creation / display means generates a serial number or the like for automatic identification, and controls the GUI screen creation / display means to position the object. 3. The man-machine interface device according to claim 1, further comprising means for superimposing the serial number or the like in a place where the serial number is to be performed. 5. The apparatus according to claim 3, wherein said character string creating / displaying means has means for lowering the luminance of the entire screen or the luminance of the background of the character portion when performing the superimposing. 2. The man-machine interface device according to 1. 6. The man-machine interface device according to claim 1, wherein a place where the keyword is superimposed is on or near the object. 7. A GUI on a display screen by a computer
(Graphical User Interface) is a recording medium that records a control program of a man-machine interface device provided with a GUI screen creation / display means capable of operating a human voice. On the other hand, voice input to the GUI screen creation / display means is O
A recording medium storing a man-machine interface control program, wherein a character string for assisting an operator is generated and displayed in conjunction with ON of a switch for N / OFF. 8. A GUI on a display screen by a computer
(Graphical User Interface) is a recording medium that records a control program of a man-machine interface device provided with a GUI screen creation / display means capable of operating a human voice. A character string for assisting the operator is generated, and the generated character string for assisting the operator is generated by the GU.
Turn on / off voice input to I screen creation / display means
A recording medium on which a man-machine interface control program is displayed, the display being interlocked with the turning on of a switch to be operated. 9. The control program causes a computer to extract keywords such as file names and tags from objects such as icons and images being displayed in response to the switch ON, and control the GUI screen creation / display means. 9. The recording medium according to claim 7, wherein the keyword is superimposed at a position where the object is located. 10. The control program causes the computer to generate a serial number or the like for automatic identification when the keyword cannot be extracted, and controls the GUI screen creation / display means to position the object. 10. The recording medium according to claim 9, wherein the serial number and the like are superimposed at a location where the program is to be performed. 11. The control program instructs a computer to perform brightness superimposition on the entire screen when performing the superimposing.
11. The recording medium according to claim 9, wherein the luminance of the background of the character portion is reduced.