JP2000184346A - Information terminal device, information communication system and display state control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device which remote operates the presentation by sending the presentation information to be shown on a display device of a terminal of the opposite party via a transmission line and then sending a command to the opposite party terminal via the transmission line to designate the display mode of the presentation information to the display device of the opposite party terminal. SOLUTION: A control part 100 controls the display states of images and data, and an input part 101 inputs the commands for a presenter, i.e., a sender of information to designate the presentation materials and also to progress the presentation. A data storage part 102 preserves the presentation materials, and an output part 103 reproduces the images and voices and also displays the presentation materials. A camera part 105 photographs the presenter himself/ herself at a presenter terminal and also photographs the audiences, a lecture hall, etc., at an audience terminal respectively. A transmission line I/F part 107 connects a physical transmission line to a system, and an enlargement display part 108 enlarges the display contents of the part 103 so as to enable the audiences to see better these contents.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information terminal device, an information communication system, and a display state control method used for a TV telephone / TV conference system and the like.

[0002]

2. Description of the Related Art Conventionally, there has been proposed a TV telephone / TV conference system for exchanging various kinds of information such as images, voices, and data between terminals via a network. As this type of TV telephone / TV conference system, reference 1 (Japanese Unexamined Patent Application Publication No.
No. 81952, “Camera Control System and Image Input Device”, and Reference 2 (Japanese Unexamined Patent Application Publication No. 9-154116, “Cursor Display Control Method of Video Conference System”) are known.

[0003] Document 1 discloses a remote control method of a camera in a video conference system. It is a very common requirement in a video conference system that the receiving side of the video actively change the shooting range by remote control. ITU-T H.264, an international standard for videoconferencing systems. 320 / H. 323 /
H. Data channel standard T.324, which is a subset of Application H.120 on H.120 281 specifies a remote camera control method.

[0004] Further, in the above-mentioned Document 2, there is a proposal on how to display a cursor on the terminal side in a video conference on the partner side. In this example, it is premised that the self image and the partner image are displayed in a picture-in-picture manner. However, the cursor as a pointing rod (pointer) is operated in cooperation between the own terminal and the partner terminal (that is, remote control is performed). Control).

[0005]

As described above, remote control of cameras and sharing of materials in a video conference are common in systems that have been commercialized and are on the market. However, most of the controls are "sender / receiver" equal or "leader" -led.
For example, in the document (1), a viewer of an image remotely controls a shooting range. Also, when sharing the materials, if the same material is developed on both sides, how to display it and what page to see etc. are decided by the operator on the terminal side that received the material, Could not control the "look".

[0006] On the other hand, in a general conference or a presentation at a conference, not in a video conference, the information sender (presenter) controls the display of the material, and the presenter determines which part of the material is pointed by the pointing stick. Is generally determined, and there is no such thing as receiving information (audience side) to control. If the recipient pre-reads the displayed materials without permission, the presentation may be hindered.

As described above, conventionally, when a remote presentation is performed using a videophone or a video conference system, there has been a problem that the terminal of the receiver cannot be controlled by the presenter as in a general presentation.

In recent years, the number of people using a combination of a personal computer and an electronic projector (projector) for presentations at conferences and conferences has been increasing, and a system suitable for presentation use has been demanded.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and is intended to improve the mechanism of a video telephone and a video conference system to remotely perform a presentation similar to a general conference or a conference. It is an object of the present invention to provide an information terminal device, an information communication system, and a display state control method capable of performing the following.

[0010]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides an information terminal device capable of transmitting and receiving information including images and data via a transmission line, and displays the information on a display device of a partner terminal. Presentation information transmitting means for transmitting the presentation information to be transmitted via the transmission path; and a pre-defined specification for designating, in the counterpart terminal, in what state the display information is to be displayed on the display device of the counterpart terminal. Command transmitting means for transmitting the transmitted command to the partner terminal device via the transmission path.

In this information terminal device, the display position of the presented information on the partner terminal and the enlargement / reduction can be controlled by command control from the information terminal device on the information sender side. Therefore, when the present invention is applied to a presentation system, it becomes possible to control the display state of the terminal on the receiver side under the initiative of the information sender, and to perform a presentation similar to a presentation at a general meeting or conference by remote control. It becomes possible.

[0012] The presenting information transmitting means may include, as the presenting information, at least one of image information of the information sender photographed on the information terminal device side and material information for a presentation by the information sender. Transmitting means, the command transmitting means, the image information,
And means for transmitting a command for controlling a display state of each of the material information, and means for transmitting cursor position indicating information for indicating a cursor position for pointing to a specific item on the material information. .

[0013] With this, the display form of each piece of information such as image information, material information, and a cursor can be controlled by the sender of the information, so that the arrangement of the information on the display screen of the receiver can be determined by the sender of the information. It is possible to specify freely by remote control.

The presenting information transmitting means includes means for transmitting, as the presenting information, image information of an information sender photographed on the information terminal device side, and the command transmitting means includes the information terminal device and the information terminal device. The display screen size ratio in the horizontal and vertical directions between the other terminal and the display position and size of the image information on the information terminal device side, based on the image information to be displayed on the other terminal. The display position and the size are specified.

When the size and ratio of the display screens of the terminals on the presenter side and the audience side are different, if absolute coordinates and the absolute value of the size are transmitted as a control command for changing the image display position, the display range on the screen on the audience side is changed. The display may be distorted from the display, or the display may be unnecessarily large, making it difficult to view the material.However, the presenter uses the horizontal and vertical size ratios between the display screen of the presenter terminal and the display screen of the audience terminal. By converting the display position and size of the image information on the side terminal according to the audience side terminal, it becomes possible to arrange the image of the presenter who is the information sender at an appropriate position on the audience side screen. In this case, the relative position of the image information on the presenter-side terminal is specified by dividing line segments in the horizontal and vertical directions of the display screen of the partner terminal by a ratio (for example, the X-axis direction of the display screen is set to 1). : Designation method of dividing into two points).

The presenting information transmitting means includes means for transmitting, as the presenting information, image information of an information sender photographed on the information terminal device side, and the command transmitting means includes a display on the other terminal. Means having a plurality of display position information corresponding to a plurality of display positions predetermined as display positions of the image information to be displayed on a screen, and selecting a predetermined display position from among the plurality of display positions And transmitting a display position information corresponding to the selected display position to the partner terminal, and indicating a display position of the image information on a display screen of the partner terminal. I do.

By preparing the display position options in advance, it is possible to control the image of the presenter only by a simple operation of selecting the display position from the options.

The presenting information transmitting means includes means for transmitting material information for a presentation as the presenting information, and the command transmitting means includes a material information displayed on a display screen of the information terminal device. Means for acquiring the relative coordinates of the cursor on the display area of the information terminal, means for acquiring the position and size of the display area of the material information for each of the information terminal device and the partner terminal, the information terminal device and the partner Relationship between the position and size of the display area of the material information between the side terminal,
On the display screen of the counterpart terminal, based on the obtained relative coordinates of the cursor, so that the information terminal device and the counterpart terminal move the cursor to the same position on the display area of the material information. Means for designating a cursor position displayed on the screen.

Thus, even if the display position and the size of the material image are different between the sender terminal and the receiver terminal, the same position on the material as the position pointed by the sender terminal at the receiver side. It becomes possible to point with the cursor.

In this case, means for holding the absolute coordinates of the cursor in the information terminal device, and a virtual cursor position indicating a virtual cursor position adapted to a coordinate system only for the display area of the material information Means for holding coordinates,
When the cursor is moved on the display area of the material information on the information terminal device side, the value of the virtual coordinate is changed according to the amount of movement, and the changed value of the virtual coordinate is changed to the absolute coordinate of the cursor. By performing the conversion again, the information terminal is moved so that a cursor is moved to a position on the display area of the material information on the information terminal device side corresponding to a cursor position on the display area of the material information on the partner terminal side. It is preferable that the apparatus further comprises means for correcting the cursor position on the device side.

Even when the minimum movement unit of the cursor is different between the sender and the receiver by reducing the display of the material on the information sender side, feedback control of the cursor movement of the presenter on the sender side is performed in this way. Accordingly, the movement of the cursor on the reduced material screen on the sender side becomes relatively slow, and the cursor on the receiver side can be moved smoothly.

According to the present invention, in an information terminal device capable of transmitting and receiving information including an image and data via a transmission path, presentation information for screen display transmitted from a partner terminal via the transmission path is provided. Presentation information receiving means for receiving;
Display state control means for receiving a command transmitted from the partner terminal via the transmission path, and changing a display state of the presentation information on a display device of the information terminal device based on the command. It is characterized by the following.

As described above, by changing the display state on the screen at the information receiving side in accordance with the instruction from the sender side, the control by the sender side can be performed.

In this case, the display state control means detects whether or not the image information is displayed on the display screen of the information terminal device when a page update of the material information is instructed from the partner terminal. In the case where the image information is displayed, it is preferable that the image information is temporarily erased and redisplayed after a predetermined time has elapsed. This makes it possible to show the entire page of the updated material information to the recipient, no matter where the image of the presenter is placed.

Further, the display state control means detects whether or not the image information is displayed on the display screen of the information terminal device when a page update of the material information is instructed from the opposite terminal. When the image information is not displayed, the image information is temporarily displayed and erased again after a lapse of a predetermined time. In this way, by temporarily displaying the image of the presenter when the page of the material information is updated, the fact that the presenter has advanced the material is transmitted to the audience, and an effect of drawing attention to the material is obtained.

In the case where the presentation information includes image information of an information sender photographed by the other party's terminal and material information for a presentation by the information sender, the display state may be changed. The control means, when moving the position of the cursor for pointing to a specific item on the material information of the information terminal according to an instruction from the other terminal, the cursor position is included in the display area of the image information It is preferable to include a unit for detecting whether or not the image information is included, and a unit for changing a display position of the image information when it is detected that the image information is included. Thereby, it is possible to show the contents of the material indicated by the cursor position. Further, instead of changing the display position of the image information, the image information may be temporarily erased and re-displayed after a lapse of a predetermined time or when the cursor position moves out of the image information display area. Good.

[0027]

Embodiments of the present invention will be described below with reference to the drawings.

The videophone and videoconference can be realized by using a dedicated terminal or by connecting peripheral devices to a general-purpose device such as a personal computer and performing software control. In the embodiment, a case where a personal computer is used will be described as an example. In addition, presentation materials include text, computer graphics, photos,
Various things such as sound can be considered, but these are not particularly limited.

The configuration and operation of the information terminal device according to the present embodiment will be described by exemplifying a case where a remote presentation is performed using a videophone / videoconference communication system.

FIG. 1 is a diagram showing an example of a hardware configuration of an information terminal device according to one embodiment of the present invention. this is,
It is a general videophone or videoconferencing system terminal with an enlarged display device added, and handles not only images and sounds but also other data. In addition, ITU-T
There are several different driving routes (GSTN / IS
DN / LAN, etc.), and respective standards are defined. However, in the present embodiment, a driving path is not particularly defined. Also, various encoding / decoding methods for each of image, audio, and data (H.261) are used.
/ H. 263, G.R. 728 / G. 723.1 / G. 72
9), but is not particularly limited. The multiplexing / demultiplexing method (H.223 / H.225) and the overall system control method (H.320 / H.323 / H.324) are not particularly limited. Further, in the present embodiment, the encoding / decoding unit, the multiplexing / demultiplexing unit, and the control unit are all assumed to be implemented by software, but may be actually implemented by hardware.

In FIG. 1, the presenter terminal (sender) and the audience terminal (receiver) are not distinguished, and the basic terminal configuration is the same. However, in an actual audience terminal, a projector and an external speaker are connected to the output unit in consideration of listening and listening by a large number of people. Further, in FIG. 1, the connection form over the transmission path is omitted, but in an actual remote presentation system, a one-to-one connection form or a one-to-many connection form is conceivable because the composition is a presenter-audience.

In FIG. 1, reference numeral 100 denotes a control device which is a center of the present system.
It performs separation processing and controls the display state of images and data.
Specifically, a CPU of a personal computer or the like corresponds to this. Reference numeral 101 denotes an input unit, which is used by a presenter who is an information sender to input a command for designating a connection destination of a videophone / conference, designating presentation materials, or proceeding with a presentation. Specifically, they are a keyboard, a mouse, a panel switch, and a remote controller.

A data storage unit 102 stores a program for controlling the entire system, a program for encoding / decoding, and a presentation material. Specifically, a hard disk drive or R
OM (Read Only Memory), DVD drive, and the like. An output unit 103 reproduces images and sounds and displays presentation materials. Specifically, it is a display, a TV, a projector, a speaker, or the like. 104 is a RAM (Random Ac)
ESS section, which is used as a work area for the system control program, temporarily stored data, and a buffer for encoding / decoding processing. Reference numeral 105 denotes a camera unit. The terminal on the presenter side photographs the presenter itself, and the terminal on the audience side photographs the audience or the lecture hall. Reference numeral 106 denotes a microphone. The terminal on the presenter side takes in the speech of the presenter into the system, and the terminal on the audience side takes in voice such as questions of the audience into the system. A transmission path interface unit 107 connects a system to a physical transmission line such as a general public line, an ISDN network, or a LAN. Specifically, it is a modem, a TA, or a LAN card, but the transmission path is arbitrary in the present embodiment. Reference numeral 108 denotes an enlarged display device, which enlarges and displays the content displayed on the output unit 103 so that it can be seen by a plurality of audiences. This is usually provided only in the terminal on the audience side, and specifically, is a combination of a screen and a projector or a large display. 1
It would not be necessary for a one-to-one presentation.

FIG. 2 shows an example of the display screen of each of the presenter terminal and the audience terminal in the remote presentation system.

The presenter image captured by the camera of the presenter terminal and the audience image captured by the camera of the audience terminal are exchanged with each other. On the display screen of the presenter-side terminal, as shown in the figure, a presenter image, audience image, presentation material, and a GUI (control) for performing control such as advancing or returning the presentation material or changing the material are displayed. From the presenter terminal to the audience terminal,
The above-described presenter image and information of the presentation material are sent to the audience terminal as presentation information to be displayed on the screen. On the display screen of the audience terminal, for example, as illustrated, a presentation material is displayed on the entire screen, and a presenter image is displayed on the material image.

Further, in the present embodiment, a screen control command is sent from the presenter side terminal to the audience side terminal so that the display screen of the audience side terminal can be controlled by the presenter. This screen control command
This is a command for instructing the audience terminal side how to display the presentation information, and includes (1) an image control command for controlling the display position and size of the presenter image, and (2) Material control commands to control the display position and size of presentation material information,
(3) It includes a cursor movement command for indicating the position of a cursor used as a pointing stick for pointing to a specific item on the presentation material image.

FIG. 3 is a block diagram showing a functional configuration of the information terminal of FIG. Here, functional blocks such as an encoding / decoding unit are collectively shown for each processing application.
This can be realized by software or hardware.

Reference numeral 200 denotes a system control unit which issues a command to each processing unit and interprets and responds to a control message or the like from the other party. Reference numeral 201 denotes a video encoding unit, which encodes an input video, compresses the video code, and sends it to the multiplexing unit 204. Reference numeral 202 denotes an audio encoding unit which encodes input audio, compresses the audio data, and sends the compressed data to the multiplexing unit 204. A data input unit 203 encodes presentation materials, system control commands from the system control unit 200, remote control commands from the presenter, and the like, and sends them to the multiplexing unit 204. A multiplexing unit 204 multiplexes a data sequence sent from 201 to 203 and multiplexes the data sequence.
Send to A transmission line I / F unit 205 connects to a transmission line for communicating with a partner terminal.
Then, the multiplexed data sequence received from 4 is sent to the other terminal. At the same time, it receives the multiplexed data string received from the other party and sends it to the separation unit 206. Reference numeral 206 denotes a separation unit, which is a multiplexing unit 2
04, that is, the multiplexed data sequence is separated into video codes, audio codes, and data, and
7. Send to the audio decoding unit 208 and the data output unit 209.
Reference numeral 207 denotes a video decoding unit, which expands the compressed video code, decodes the code, and returns it to video data. Reference numeral 208 denotes an audio decoding unit that expands the compressed audio code, decodes the code, and returns the audio data to audio data. Reference numeral 209 denotes a data output unit that generates presentation materials from a data string received from the partner terminal, interprets system control commands and remote control commands, and sends them to the system control unit 200.

FIG. 4 is a flowchart showing the basic operation of the present system.

Here, the images and images are mutually exchanged as a videophone.
A state in which voice communication is performed will be referred to as a "normal mode", and one of the states will be referred to as a presenter, and a state in which materials are displayed or a cursor as a pointing stick is remotely operated will be referred to as a "presentation mode".

FIG. 4 shows a processing flow from the state before communication to the communication state, from the normal mode to the presentation mode, back to the normal mode, and to the end of the communication. "Presentation mode"
Will be described in detail with reference to another figure for each function.

In FIG. 4, S01 is a step for performing a system initialization process. Here, the transmission path I / F unit 10
7 and control programs stored in the data storage unit 10
2 to the RAM unit 104. S02 is a step of performing outgoing / incoming processing for connection between terminals. If it is a public line, enter a telephone number to make a call or receive a call to make a connection. S03 is a step of performing a communication initialization process. Here, the capability information of the terminals is exchanged, and a logical channel for exchanging video / audio data is established. S04 is a step of starting encoding / decoding of video / audio. At this point, a videophone call starts between the terminals (normal mode). In step S05, the call state of the system is changed from "normal mode" to "presentation mode" depending on whether or not to start the presentation.
This is the step to shift to. If the presentation is to be started, the process proceeds to S06, the presenter side terminal sends a presentation start request to the audience side terminal, and the communication state of the system shifts to "presentation mode". Otherwise, it remains in the "normal mode" and proceeds to S08.

Step S06 is a step of performing "presentation mode processing". Here, the user of the presenter-side terminal, that is, the presenter, sends materials for presentation to the audience-side terminal, which is the recipient side, controls the display state of various information sent to the audience-side terminal, and remotely controls cursor movement. . Details will be described later.

Step S07 is a step of returning the communication state of the system from "presentation mode" to "normal mode" depending on whether or not to end the presentation. When ending the presentation, the mode shifts to the “normal mode” and the process proceeds to S08. In the case of continuation, the process returns to S06. S08 is a step of selecting whether to end the call. If the call is to be ended in response to a user request or a request from the partner terminal, the process proceeds to S09,
If not, the process returns to S05. S09 is a step of stopping the encoding / decoding processing of the image / audio. S10
Is a step of performing call termination processing, in which a logical channel is closed or a call is disconnected. S11 is a step of determining whether to end the system.

[Detailed Description of Presentation Mode]
Hereinafter, step S06 in the flowchart of FIG.
That is, the details of the “presentation mode” will be described. In the "presentation mode", materials can be transferred from the presenter-side terminal and displayed on the audience side, or the cursor of the audience-side terminal can be remotely controlled from the presenter side. At this time, there are two things that the presenter wants to display on the audience terminal, one is a presentation material, and the other is a moving image of himself (presenter) photographed by a camera. Transmission of material data and remote control commands from the presenter in the “presentation mode” are exchanged using the data channels of the data input unit 203 and the data output unit 209 in FIG.

FIG. 5 is a flowchart in the “presentation mode”.

Step S21 is a step of performing a presentation initialization process. The properties of the material file used for the presentation from the presenter terminal to the audience terminal (file name, size, date and time of creation, etc.)
Is sent. In step S22, the viewer terminal transmits the properties of the file sent from the presenter terminal and the name and size of the file stored in its own data storage device.
This is a step of comparing the creation date and time and sending a transmission request from the audience terminal to the presenter terminal in the case of a material file that is missing or has a mismatched size.
S23 is a step of transferring the material file. FIG. 10 shows how such property information comparison and file transfer are performed. Thereby, it is possible to eliminate waste such as transferring a file already at hand.

S24 is a presentation start process, in which a material file is developed on the terminal of the presenter, and a control which is a GUI for performing control such as advancing or returning the material and changing the material is also displayed.
On the other hand, at the audience terminal, only the material file is expanded, and the control is not displayed.

In step S25, the presenter inputs a command for performing remote control of the presentation. To explain the command types, first,
There is an image control command for the presenter to remotely control his / her own image display state on the audience side. Using this, the presenter enlarges or reduces his or her own image, and changes the display position. Next, there is a material control command for controlling the display state of the material file for presentation. Since the material file holds and presents information composed of a plurality of pages, and generally proceeds forward or returns to the previous page, in the present embodiment, “forward / previous Control with three commands of "Return to / Change material"
It is displayed on the presenter's terminal. Note that instead of displaying the controller, the keyboard displays
The same can be achieved by mapping the function of "return to previous / change of material". There is also a cursor control command for the presenter to remotely control the cursor of the audience terminal. In S25, the process branches to the next step according to the command input by the presenter. The process branches to S26 in the case of an image control command, to S28 in the case of a material control command, and to S30 in the case of a cursor control command.

(1) Image control command S26 is a step of transmitting an instruction for the display state of the presenter screen. The presenter selects how to display his / her own image on the audience terminal, and in accordance with the selection, the presenter terminal sends a command for performing display control to the audience. In step S27, the audience terminal updates the display. The command transmitted in S26 is received, the content of the command is interpreted, and the display state of the presenter image is updated according to the instruction. The content of the command to be transmitted is a code indicating that the command is an image control command, and a status indicating parameter group following the code. Parameters necessary for defining the display state (display state instruction parameters) are a display image size (width, height), a display position (x, y coordinates on the screen), and ON / OFF of display. This example is shown in FIG. FIG. 6A shows a display screen displayed on the display device of the audience terminal, and FIG. 6B shows display state instruction parameters. When the display is set to ON by the display state instruction parameter, the presenter image is displayed at the position designated by the display position (x, y) as shown in the figure, with the display image size (width width, height h).
(right).

These three types of parameters (display position,
(Size, display ON / OFF) will satisfy the request of the presenter, but the display screen (display screen) size of the sender and the receiver does not always match, so the relative coordinates and the overall screen size It is preferred to deliver using a ratio to This is shown in FIG. FIG. 7A shows the correspondence between the display screen (screen) of the presenter terminal and the display screen (screen) of the audience terminal. The display position (Xa, Ya) of the presenter image instructed from the presenter terminal to the audience terminal is based on the absolute coordinates (Xp, Ya) of the presenter image on the presenter terminal.
Yp) The horizontal and vertical line segments of the display screen are divided so that the relative positional relationship of the presenter image with respect to the entire display screen of the presenter terminal instead of the presenter terminal is correctly reflected in the display position on the audience terminal. This is performed by an instruction using a ratio such as designating a position. For example, designation is made such that the X-axis direction of the display screen is divided into 1: 2 points.

Specifically, as shown in FIG. 7B, the position Xa on the line segment in the X-axis direction of the audience terminal is set to the value of Xp in the X-axis direction of the presenter terminal. Is multiplied by the ratio of the display screen size (Width-P) of the audience terminal in the X-axis direction to the display screen size in the X-axis direction of the audience terminal (Width-A). It can be specified as a ratio to the size (Width-A). Similarly, for the position Ya on the line segment in the Y-axis direction of the audience terminal, the display screen size in the Y-axis direction of the presenter terminal (High-P) is set to Yp.
And the display screen size (High) of the audience terminal in the Y-axis direction.
-A), the value of Yp is changed to the display screen size (High-
It can be specified as a ratio to A). . The display image size (width, height) of the presenter image instructed to the presenter terminal is also the display image size (width-0, height high) of the presenter image on the presenter terminal.
At t-0), the ratio of the display screen size (Width-P) of the presenter terminal in the X-axis direction to the display screen size (Width-A) of the audience terminal in the X-axis direction, and the Y-axis of the presenter-side terminal Display screen size in direction (Hig
ht-P) and the display screen size (High-A) in the Y-axis direction of the audience terminal are multiplied to correspond to each other, so that the display image sizes width-0 and hight-0 of the presenter image are respectively changed. Display screen size in the Y-axis direction of the presenter-side terminal (High
−P) and the ratio to the display screen size (High-A) in the Y-axis direction of the audience terminal.

As described above, the display position and the size of the presenter image on the presenter side terminal are used to determine the display position and size of the presenter image on the presenter side terminal using the horizontal and vertical size ratios between the display screen of the presenter side terminal and the display screen of the audience side terminal. By performing the conversion according to the display screen of the terminal, it becomes possible to arrange the presenter image at an appropriate position on the screen on the audience side.

The size information in the horizontal and vertical directions of the display screen of the audience terminal may be obtained from the audience terminal at the time of the initialization processing in S21, for example.

In the presentation, rather than changing the size and position of the presenter image subtly,
A rough state change, such as "large display / small display" or "display in the upper right / display in the lower right", can be more easily instructed and useful. FIG. 8 shows an example of a controller for transmitting an image control command on the presenter side. The following six options are available. This is sufficient for a simple configuration.

1. 1. Small display in upper left 2. Small display in the upper right corner 3. Small display at lower left 4. Small display at lower right Large display at center 0. No display (screen is erased) The display state (display position, size) corresponding to each of these six options is previously determined with the audience terminal, so that the display state requested by 3-bit information can be uniquely determined. Can decide. Therefore, it is possible to reduce the data amount of the transmission command. After updating the image display state in the audience terminal, the process proceeds to S50.

(2) Material control command S28 is a step of sending a presenter's instruction for the display state of the material as a control command. There are two commands for an instruction for the currently displayed material: “advance / return”. This command is used to advance or return the display page of the material by one, or to advance or return the effect of the material by one step. For example, as shown in FIG. 9, when a command of “advance” is transmitted from the presenter side terminal to the audience side terminal while the second page of the material is displayed on the audience side terminal,
The material displayed on the audience terminal is updated to the third page. Of course, the material is also updated to the third page in the presenter-side terminal.

When the presenter changes the material, the presenter sends the property information of the material file. If there is no material matching the property in the audience terminal, the file is transferred (FIG. 10). S29 is a step of updating the display state of the material at the audience terminal or changing the display material. Thereafter, the process proceeds to S32.

(3) Cursor Movement Command S30 is a step of sending a cursor control command from the presenter. When the presenter traces over the material displayed on his / her own terminal with the cursor, the presentation is easy if the audience reflects it. If a simple method is selected, during the presentation mode, the cursor position information (x, y coordinates on the screen of the presenter-side terminal) is regularly acquired and sent to the audience. However, in this case, the screen state of the presenter side terminal and the screen state of the audience side terminal need to match, but the presenter side also needs to input commands such as (1) and (2) described above. It is also required that the presentation manuscript be secretly displayed, so that it is desirable to implement cursor control in a separate display state rather than matching with the audience. Also, it is important to stop the cursor on the audience side at a certain position on the material, so that the presenter can start remote control of the cursor by selecting “ON / OFF of the cursor control mode”. And be able to stop. In the present embodiment, FIG.
When the presenter moves the cursor over the reduced display area and the presenter has set the cursor to the "cursor control mode ON" (for example, when the cursor is While pressing a key on the keyboard,
Only), the cursor position coordinates on the presenter side are acquired, and the cursor position on the material is converted into relative coordinates or ratios and sent to the audience side. That is, while acquiring the relative coordinates of the cursor on the material display area on the presenter-side terminal, the position and size of the material display area on each of the presenter-side terminal and the audience-side terminal are acquired, and the relative coordinates, the presenter-side terminal and the audience side are acquired. The cursor position is indicated by expressing the relative coordinates as a ratio to the line segments in the X-axis and Y-axis directions of the screen of the audience terminal based on the relationship between the position and size of the material display area with the terminal. do it. Thus, the presenter-side terminal and the audience-side terminal can be controlled so that the cursor is moved to the same position on the material display area.

When the display size of the material is different between the presenter side and the audience side as described above, the smaller the size of the presenter side, the more the cursor remote control result looks awkwardly moving at the audience side. . This is because the minimum unit of cursor movement on the presenter side is amplified on the audience side and corresponds to a large movement distance. To avoid this, it is necessary to slow down the cursor movement on the material on the presenter side, that is, to match the cursor movement on the material on the presenter terminal side with the movement unit on the audience side only on the material. Specifically, the screen coordinates (X,
Apart from Y), there is a variable (Xa, Ya) for holding the relative cursor position coordinates on the material, and when the cursor moves on the material, the variable is changed and converted to the actual screen coordinates. Feedback control of the cursor position.

For example, if the data on the presenter side is (p,
q), it is assumed that the size is width Wa and height Ha. It is also assumed that the same material is displayed at the position of the coordinates (0, 0) on the screen with the width W and the height H on the audience side. Here, when the relative cursor position coordinates on the material on the presenter side is the upper left corner of the material display area, that is, Xa = x0, Ya = y0, the cursor display position on the audience side is the upper left corner of the display screen ( (x0, y0), the screen coordinates of the cursor on the presenter side are set. The calculation formula is: X = p + Xa · Wa / W Y = q + Ya · Ha / H (Formula 1) Here, it is assumed that the cursor is moved by the presenter, and the screen coordinates are displaced in the screen coordinates as X → X + dx Y → Y + dy on the presenter image. The cursor moves from (X, Y) to (X + d
(x, Y + dy). In some cases, it may be off the presentation material. Therefore, the cursor position is corrected by feedback control. That is, since the OS (operating system) of the presenter-side terminal directly counts the cursor displacement dx, dy, the cursor of the presenter-side terminal normally moves by the displacement dx, dy. Since the material display area is smaller than the audience side due to the reduced display of the material, control for returning the cursor position by that amount is required. For example, when the material is displayed in a reduced size of 1/5, it is necessary to correct the movement amount of the cursor so that it becomes 1/5 of the actual displacement amount (the displacement amount counted by the OS). .

That is, in this case, the cursor position coordinate variables (Xa, Ya) for the material are dx and dy, respectively.
, The screen coordinates are calculated again by (Equation 1), and the cursor may be moved there. The screen coordinates of the destination are (p + (Xa + dx) · Wa / W, q + (Ya + dy)
・ Ha / H). Of course, after this feedback, Xa = Xa
+ Dx, Ya = Ya + dy.

Step S31 is a step of updating the cursor display position on the audience side. The audience moves the display position of the cursor according to the received relative coordinates or ratio. Thereafter, the process proceeds to S32.

As described above, according to the cursor movement amount counted by the OS, the cursor position coordinate variable (X
a, Ya) is changed, and the changed value of the cursor position coordinate variable (Xa, Ya) is converted back to the absolute coordinates of the cursor according to (Equation 1). The movement of the cursor can be made slower than in the case of the area above, and the unit of movement of the cursor on the material display area can be made the same on the presenter side and the audience side. Therefore, even when the material is reduced on the presenter side, the position on the reduced material display area that is the same as the position on the material specified by the cursor moving destination on the audience side can be specified with the cursor on the presenter side At the same time, the cursor on the audience side can be smoothly and remotely controlled.

Step S32 is a step of determining whether or not to end the presentation mode. When there is a termination request from the presenter, a control command instructing return to the “normal mode” is transmitted to the audience terminal, and the process returns to the “normal mode”. S33 is a step of performing a presentation end process, such as erasing the controller for the presentation, stopping the display of the presentation material, and returning the display position of the image to the state before the presentation.

The remote presentation is realized according to the above flow. By the way, images, materials,
Several problems remain with respect to cursor display state control. For example, "a part of the material is hidden by the presenter image, and the presenter points with the cursor, but the location is not visible", or "always cares about the presenter image and does not pay attention to changes in the material" This is a problem on the audience side. Of course, these dissatisfactions can be reduced by consideration from the presenter side, but it is more preferable to avoid such dissatisfaction by adding a unique judgment criterion and automatic processing to the audience terminal. Hereinafter, the problem will be divided into two, and a method of solving the dissatisfaction will be described.

[When the Cursor Moves on the Presenter Image] In the case where the region of the material that the presenter wants to point to is hidden by the presenter screen on the audience side (FIG. 12A), The terminal moves the display screen of the presenter to another place or temporarily deletes the presenter screen (FIG. 12B). As a method, the audience terminal always evaluates whether the cursor display update position indicated by the cursor control command is included in the display area of the presenter screen, and if it is included, another position (for example, lower right → upper left, etc.) Move the presenter screen or erase the presenter screen.

[If the Display Page of the Material Has Changed] When the presenter has advanced the presentation as described in step S28, it is more convenient to communicate this to the audience. Therefore, in order to obtain the effect of attracting attention to the material, when the page update of the material is instructed from the presenter-side terminal, that is, when the material at the audience terminal advances to the next page, FIG. In this way, the presenter screen is deleted for a certain period of time from the page update time, and is redisplayed after a certain period of time as shown in FIG. This is effective when the presentation is proceeding in a state where the presenter screen is displayed. On the other hand, when the presentation is proceeding in the “state in which the presenter screen is erased”, displaying the presenter screen only for a certain period of time from the time of page update has an effect of attracting the audience's attention. In other words, in this case, when the page update of the material is instructed from the presenter terminal, that is, when the material at the audience terminal advances to the next page, the presenter screen is displayed for a certain period of time from the time of page update, and It may be deleted after a lapse of time.

As described above, in the present embodiment, it is possible to remotely control the screen display of the receiver under the control of the sender, and to perform a remote presentation similar to that of a general conference or conference. It can be performed by operation.

Since all of the above-described processing procedures can be realized by software, if a computer program for executing the procedures is recorded on a computer-readable recording medium, the computer program can be transferred to a computer with a camera through the recording medium. The same effect as in the present embodiment can be obtained simply by introducing the present invention to a portable information terminal or the like.

[0071]

As described above, according to the present invention,
It is possible to remotely control the screen display of the receiver side under the control of the sender side, and a presentation similar to a presentation at a general conference or a conference can be made on a TV phone / TV.
It can be realized by a conference type system.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an example of a hardware configuration of an information terminal device used in a remote presentation system according to an embodiment of the present invention.

FIG. 2 is an exemplary view showing an example of a display screen of each of a presenter side terminal and an audience side terminal in the remote presentation system of the embodiment.

FIG. 3 is a block diagram showing a functional configuration of the information terminal shown in FIG. 1;

FIG. 4 is an exemplary flowchart showing the basic operation of the remote presentation system of the embodiment.

FIG. 5 is an exemplary flowchart illustrating a processing procedure in a “presentation mode” of the remote presentation system according to the embodiment.

FIG. 6 is an exemplary view for explaining image control commands used in the remote presentation system of the embodiment.

FIG. 7 is an exemplary view for explaining an example of an image control command applied when the display screen size is different between the presenter side terminal and the audience side terminal used in the remote presentation system of the embodiment.

FIG. 8 is an exemplary view showing an example of a control GUI used in the remote presentation system of the embodiment.

FIG. 9 is an exemplary view showing the state of progress of the presentation in the remote presentation system of the embodiment.

FIG. 10 is an exemplary view showing a state of comparing property files and transferring material files in the remote presentation system of the embodiment.

FIG. 11 is an exemplary view showing an example of remote control of a cursor in the remote presentation system of the embodiment.

FIG. 12 is an exemplary view for explaining display control processing performed in the audience terminal of the remote presentation system of the embodiment.

FIG. 13 is an exemplary view for explaining another example of the display control process performed by the audience terminal of the remote presentation system of the embodiment.

[Explanation of symbols]

100 control unit, 101 input unit, 102 data storage unit, 103 output unit, 104 RAM unit, 105 camera unit, 106 microphone unit, 107 transmission line I / F unit, 2
00: system control unit, 201: video encoding unit, 202
... Speech coding unit, 203 ... Data input unit, 204 ... Multiplexing unit, 205 ... Transmission path I / F unit, 206 ... Separation unit, 20
7: video decoding unit, 208: audio decoding unit, 209: data output unit.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) H04M 11/06 H04M 11/06 F term (Reference) 5C064 AA02 AB03 AB06 AC07 AC13 AD14 5C082 AA01 AA03 AA21 AA27 AA34 AA36 BA02 BA12 BA41 BB01 CA02 CA33 CA34 CA52 CA64 CB05 DA51 DA87 MM09 MM10 5K101 KK04 NN18 TT06

Claims (15)

[Claims] 1. An information terminal device capable of transmitting and receiving information including an image and data via a transmission line, wherein presentation information transmission means for transmitting presentation information to be displayed on a display device of a partner terminal via the transmission line. And a predetermined command for specifying the presentation information on the display device of the partner terminal in what state the screen is to be displayed on the partner terminal, to the partner terminal device via the transmission path. An information terminal device comprising command transmission means for transmitting. 2. The presenting information transmitting means, as the presenting information, at least one of image information of the information sender photographed on the information terminal device side and material information for a presentation by the information sender. Transmitting means for transmitting a command for controlling a display state of each of the image information and the material information; and instructing a cursor position for pointing to a specific item on the material information. 2. The information terminal device according to claim 1, further comprising: means for transmitting cursor position indication information. 3. The presenting information transmitting means includes means for transmitting image information of an information sender photographed on the information terminal device side as the presenting information, wherein the command transmitting means includes the information terminal device and the information terminal device. It should be displayed on the partner terminal based on the display screen size ratio in the horizontal and vertical directions with respect to the partner terminal and the display position and size of the image information displayed on the information terminal device side. The information terminal device according to claim 1, wherein a display position and a size of the image information are specified. 4. The command transmitting means determines a relative position of the image information in the horizontal and vertical directions on the information terminal device side by dividing a line segment in a horizontal and vertical direction of a display screen of the counterpart terminal into a ratio. 4. The information terminal device according to claim 3, wherein the information terminal is instructed. 5. The presenting information transmitting means includes means for transmitting, as the presenting information, image information of an information sender photographed on the information terminal device side, wherein the command transmitting means includes a display of the counterpart terminal. Means having a plurality of display position information corresponding to a plurality of display positions predetermined as display positions of the image information to be displayed on a screen, and selecting a predetermined display position from among the plurality of display positions And transmitting a display position information corresponding to the selected display position to the other terminal, and instructing a display position of the image information on a display screen of the other terminal. The information terminal device according to claim 1. 6. The presentation information transmitting means includes means for transmitting material information for a presentation as the presentation information, and the command transmitting means includes material information displayed on a display screen of the information terminal device. Means for acquiring the relative coordinates of the cursor on the display area of the information terminal, means for acquiring the position and size of the display area of the material information for each of the information terminal device and the partner terminal, and the information terminal device and the partner The relationship between the position and size of the display area of the material information between the side terminal and the obtained relative coordinates of the cursor, based on the information terminal device and the other terminal on the display area of the material information Means for indicating a cursor position displayed on the display screen of the other terminal so that the cursor is moved to the same position of the other terminal. Terminal equipment according to claim 1, wherein. 7. A means for holding absolute coordinates of a cursor in the information terminal device, and means for holding virtual coordinates indicating a position of a virtual cursor adapted to a coordinate system targeting only a display area of the material information. When the cursor is moved on the display area of the material information on the information terminal device side, the value of the virtual coordinates is changed according to the amount of movement, and the changed value of the virtual coordinates is set to the absolute value of the cursor. By converting back to coordinates, the cursor is moved to a position on the display area of the material information on the information terminal device side corresponding to the cursor position on the display area of the material information on the partner terminal side, 7. The information terminal device according to claim 6, further comprising: means for correcting a cursor position on the information terminal device side. 8. An information terminal device capable of transmitting and receiving information including an image and data via a transmission path, wherein presentation information for receiving presentation information for screen display transmitted from a partner terminal via the transmission path. Receiving means; and a display state control means for receiving a command transmitted from the partner terminal via the transmission path, and changing a display state of the presentation information on a display device of the information terminal device based on the command. An information terminal device comprising: 9. The presentation information includes image information of an information sender photographed by the partner terminal and material information for a presentation by the information sender. Detecting whether or not the image information is displayed on a display screen of the information terminal device when an instruction to update the page of the material information is issued from the partner terminal, and if the image information is displayed, detecting the image information 9. The information terminal device according to claim 8, wherein the information terminal device is temporarily deleted and redisplayed after a lapse of a predetermined time. 10. The presentation information includes image information of an information sender photographed by the counterpart terminal and material information for a presentation by the information sender. Detecting whether or not the image information is displayed on a display screen of the information terminal device when an instruction to update the page of the material information is issued from the partner terminal, and when not displaying the image information, 9. The information terminal device according to claim 8, wherein the information terminal is temporarily displayed, and is erased again after a predetermined time has elapsed. 11. The presentation information includes image information of an information sender photographed by the counterpart terminal and material information for a presentation by the information sender. When moving the position of a cursor for pointing to a specific item on the material information of the information terminal according to an instruction from the counterpart terminal, whether the cursor position is included in the display area of the image information 9. The information terminal device according to claim 8, further comprising: means for detecting the image information; and means for changing a display position of the image information when it is detected that the image information is included. 12. The presentation information includes image information of an information sender photographed by the counterpart terminal and material information for a presentation by the information sender. When moving the position of a cursor for pointing to a specific item on the material information of the information terminal according to an instruction from the counterpart terminal, whether the cursor position is included in the display area of the image information 9. A method according to claim 8, further comprising: means for detecting the image information; and means for temporarily deleting the image information when it is detected that the image information is included and redisplaying the image information after a predetermined time has elapsed. Information terminal device. 13. The presentation information includes image information of an information sender photographed by the other party's terminal and material information for a presentation by the information sender. When moving the position of a cursor for pointing to a specific item on the material information of the information terminal according to an instruction from the counterpart terminal, whether the cursor position is included in the display area of the image information Means for detecting the image information, and when it is detected that the image information is included, the image information is temporarily deleted, and the image information is redisplayed when the cursor position moves out of the display area of the image information. 9. The information terminal device according to claim 8, further comprising means for performing. 14. In a videophone type information communication system for transmitting and receiving information including images and data between information terminals via a transmission path, presentation information to be displayed on a display device of a receiving terminal is transmitted through the transmission path. Means for transmitting from the transmitting terminal to the receiving terminal through a predetermined command for specifying to the receiving terminal how to display the presentation information on the display device of the receiving terminal. Means for transmitting from the transmitting terminal to the receiving terminal via the transmission path; and displaying presentation information displayed on the display device of the receiving terminal in accordance with a command from the transmitting terminal. An information communication system comprising: means for changing a state. 15. A display state control method applied to a videophone type information communication system for transmitting and receiving information including an image and data between information terminals via a transmission path, wherein the information is to be displayed on a display device of a receiving terminal. Transmitting the presentation information from the transmitting terminal to the receiving terminal via the transmission path; and receiving the terminal information as to how to display the presentation information on a display device of the receiving terminal. Transmitting a predetermined command for designating to the receiving terminal from the transmitting terminal to the receiving terminal via the transmission path, and displaying the command on the receiving terminal according to the command from the transmitting terminal. Changing the display state of the presentation information displayed on the device.