JP2009021922A - Video conference apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video conference apparatus which displays not only an audio or video image, but also incidental information for a speaker, at the same time. <P>SOLUTION: The video control section 13 of the video conference apparatus 1 assigns video data 501-503 of cameras CA1-CA3 to partial video regions 511-512 of composite video data 500. The composite video data 500 are constituted of four divided screen and pieces of speaker individual information 541-546 are assigned to a remaining partial video region 514. Camera identification marks 521-523 are assigned to each of the partial video regions 511-513. When speaker azimuth camera information is accepted, the video control section 13 performs setting, to display the speaker individual information and the camera identification mark, corresponding to the speaker azimuth camera information in a display mode which is different from non-corresponding speaker individual information and camera identification marks. The processing is carried out at predetermined intervals; and when a speaker azimuth is changed, the display mode of the speaker individual information and the camera identification mark is also changed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video conference apparatus that conducts a conference by communicating video and audio between conference rooms at positions separated from each other.

Conventionally, there are various types of audio conference systems that communicate audio as a system for conducting a conference between remote locations, but video conference systems that communicate video together with audio are becoming widespread. In such a video conference system, a video conference device and a display are installed in each conference room, and images taken in other conference rooms are displayed. Then, when the own device is connected to a plurality of video conference devices, videos respectively captured by the other video conference devices are displayed on the display. For example, in the multiple video composition method disclosed in Patent Document 1, each video captured in a plurality of video conference rooms is converted into a video signal matrix, and a composite video based on the video signal matrix is generated. Thereby, for example, images captured by different video conference apparatuses are assigned to and displayed on each partial area set by dividing the screen into four.
JP-A-6-30337

  However, in the conventional method as shown in Patent Document 1, a speaker can be identified by sound and video, but additional information such as the name of the speaker cannot be identified at a glance. .

  Therefore, an object of the present invention is to provide a video conference apparatus that can simultaneously identify not only audio and video but also other incidental information for a speaker.

  The video conference apparatus according to the present invention has a plurality of microphones for collecting sound and generating collected sound data as sound collecting functions, and focusing on different directions based on the collected sound data of the plurality of microphones. Sound collecting beam sound data generating means for generating a plurality of sound collecting beam sound data having the sound collecting directivity. In addition, the video conference apparatus, as an imaging function, an imaging unit that generates a plurality of video data each having a different imaging area, and a composite video that generates a composite video data by combining the plurality of video data from the imaging unit Data generating means. Furthermore, the video conference apparatus includes communication control means for outputting the collected sound beam audio data and the synthesized video data as a communication function.

Further, the video conference apparatus includes a speaker orientation detection unit that detects the speaker orientation based on the levels of a plurality of collected beam sound data.
The composite video data generation means of the video conference apparatus generates the composite video data by executing the following processing.
The composite video data generation means assigns a plurality of video data to each area obtained by dividing the composite video data into screens. The composite video data generation means is a story in which information indicating a speaker to be imaged is set in advance for each of a plurality of video data output from the imaging means (for example, a plurality of video data acquired by one or a plurality of cameras). The person display data is allocated to an area where a plurality of video data of the composite video data is not allocated. The composite video data generating means applies imaging region specifying marks (for example, camera specifying marks for each camera) individually corresponding to each video data in the screen division region of each video data. Then, the synthesized video data generating means displays the speaker display data and the imaging area specifying mark corresponding to the speaker orientation detected by the speaker orientation detecting means, the speaker display data and the imaging area specifying other than the speaker orientation. Set the display mode different from the mark.

  In this configuration, a conference person in a different imaging area is imaged by one or a plurality of cameras. When the conference speaker speaks, the direction of the conference speaker who speaks, that is, the speaker orientation is detected. For example, when there are a plurality of cameras, a camera including this speaker orientation in the imaging region is selected, and information about the selected cameras (for example, the center camera (C), the right camera (R), the left camera (L), etc.) ) Or the speaker-related information related to the speaker imaged by the camera is displayed in a display manner different from the information related to the other camera and the speaker-related information. This process is repeated at a predetermined frame rate set in advance, and the synthesized video data is sequentially output.

  In the video conference apparatus of the present invention, the speaker display data includes a plurality of individual speaker display data. The synthesized video data generation means of the video conference apparatus sets the individual speaker display data corresponding to the speaker orientation in the speaker display data based on the speaker orientation in a further different display mode.

  In this configuration, when a plurality of conference persons are imaged in one imaging area, if one of the conference persons in the imaging area becomes a speaker, not only the imaging area but also a specific area in the imaging area is selected. Composite video data that can further identify the speaker is generated.

  In the video conference apparatus of the present invention, the sound collecting beam sound data generating means selects and outputs sound collecting beam sound data corresponding to the speaker direction.

  In this configuration, the voice of the speaker is output at a high S / N ratio together with the video that can be identified by the speaker.

  The video conference apparatus of the present invention further includes a function for receiving sound emission sound data and a sound emission function. The sound emission function includes a plurality of speakers and sound emission control means. The communication control means receives the sound emission sound data corresponding to the sound collection beam sound data generated by the counterpart device and the speaker orientation associated with the sound emission sound data. The sound emission control means sets the sound emission directivity based on the sound emission sound data received by the communication control means and the speaker orientation associated with the sound emission sound data. The sound emission control means generates a speaker drive signal for each of the plurality of speakers according to the sound emission directivity. The plurality of speakers emit sound based on the input speaker driving signal.

  In this configuration, when the collected sound beam sound data as described above is transmitted from the counterpart device, the video conference device on the receiving side can perform sound image localization according to the speaker orientation. As a result, the speaker can be easily identified and more realistic speaker speech can be reproduced.

  According to this invention, not only the voice of a plurality of conference persons including a speaker voice and a speaker, but also information for identifying a speaker who is speaking is output together with the conference person video. The conference person of the other video conference apparatus that has emitted and displayed can easily identify the speaker while listening to the voice of the speaker.

A video conference apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is an external perspective view of the video conference apparatus according to the present embodiment. In the video conference apparatus according to the present embodiment, a microphone array installed on the front wall and a cover made of a punching mesh or the like are installed in front of the speaker array.
As shown in FIG. 1, the video conference apparatus 1 includes a sound emitting and collecting element installation housing and a control system circuit installation housing having a substantially long shape. The sound emitting and collecting element installation housing has two side surfaces along the longitudinal direction as a front wall and a back wall, respectively, microphones MC1 to MC16, speakers SP1 to SP14, and cameras CA1 to CA3 are installed on the front wall, and on the back wall side. A control circuit installation housing is installed.

  The microphones MC <b> 1 to MC <b> 16 have the same mechanism and the same sound collection characteristics, and are installed on the top surface side of the front wall at a predetermined interval along a straight line along the longitudinal direction. The microphones MC1 to MC16 are arranged at a narrow pitch near the center in the arrangement direction (that is, near the center when viewed from the front direction), and are arranged at a wide pitch near both ends in the arrangement direction. The microphones MC <b> 1 to MC <b> 16 are installed so that the sound collecting range is on the outer side from the front wall. Such microphones MC1 to MC16 form a microphone array whose front direction is the sound collection range.

  The speakers SP <b> 1 to SP <b> 14 have the same mechanism and the same sound emission characteristics, and are installed at a central position in the vertical direction of the front wall at a predetermined interval along a straight line along the longitudinal direction. The speakers SP1 to SP14 are installed so that the outer side from the front wall is the sound emission direction. Such speakers SP1 to SP14 form a speaker array having a sound emission range in the front direction.

The cameras CA1 to CA3 have the same mechanism and the same imaging characteristics.
The camera CA1 is installed at the center position in the longitudinal direction (that is, the center position viewed from the front direction) on the bottom side of the front wall. The camera CA1 is installed so that the center direction of the imaging range is perpendicular to the front wall.
The camera CA2 is installed at one end in the longitudinal direction on the bottom side of the front wall (in FIG. 1, the right end when the video conference device 1 is viewed from the front). When the center direction of the imaging range is a predetermined angle with respect to the front wall, for example, FIG. 1, the camera CA2 faces a predetermined position from the right end of the front wall to the front of the left end when the video conference device 1 is viewed from the front. It is set to an angle.
The camera CA3 is installed at one end in the longitudinal direction on the bottom side of the front wall (in FIG. 1, the left end when the video conference device 1 is viewed from the front). When the center direction of the imaging range is a predetermined angle with respect to the front wall, for example, FIG. 1, the camera CA3 faces a predetermined position from the left end of the front wall to the right front in a state where the video conference device 1 is viewed from the front. It is set to an angle. The imaging range of the camera CA2 and the imaging range of the camera CA3 are set as targets with the central direction of the imaging range of the camera CA1 as a reference axis. The imaging ranges of these cameras CA1 to CA3 are set so that the entire circumference along the longitudinal direction is imaged on the front side of the video conference device 1 by making each imaging range continuous.
In the control system circuit installation housing of the video conference apparatus 1, the above-described microphone array by the microphones MC1 to MC16, the speaker array by the speakers SP1 to SP14, and the functional units other than the cameras CA1 to CA3 are installed.

FIG. 2 is a diagram illustrating a functional block configuration and a connection relationship with the outside of the video conference apparatus 1 according to the present embodiment.
As shown in FIG. 2, the video conference device 1 includes a main control unit 10, a sound collection control unit 11, an echo canceller 12, and a video control unit together with the microphones MC <b> 1 to MC <b> 16, speakers SP <b> 1 to SP <b> 14, and cameras CA <b> 1 to CA <b> 3. 13, a sound emission control unit 14, a communication control unit 15, and an operation unit 16.

  The main control unit 10 performs overall control of the video conference apparatus 1 and performs control according to the operation content input by the operation unit 16. The main control unit 10 acquires selection information of the collected sound beam sound data selected by the sound collection control unit 11 and detects the speaker orientation. The main control unit 10 selects a camera whose imaging range is the detected speaker orientation, and outputs the selected camera to the video control unit 13 as speaker orientation camera information.

  Further, the main control unit 10 gives speaker direction data indicating the detected speaker direction to the communication control unit 15, and transmits the data in association with output voice data including sound collection beam voice data corresponding to the speaker direction data. The control to be performed is given to the communication control unit 15. In addition, the main control unit 10 obtains speaker orientation data associated with the output audio data of the other party's video conference device received by the communication control unit 15 and performs sound source localization according to the speaker orientation data. Is given to the sound emission control unit 14.

  The microphones MC <b> 1 to MC <b> 16 collect sound from the front side of the own device (the video conference device 1), generate a sound collecting signal, and output it to the sound collecting control unit 11.

  The sound collection control unit 11 performs signal processing with different delay processing and amplitude processing patterns on the sound collection signals of the microphones MC1 to MC16, so that different orientations are set as the central directions of the sound collection directivities. A plurality of collected beam sound data is generated. More specifically, the sound collection control unit 11 amplifies the sound collection signals of the microphones MC1 to MC16 with a predetermined amplification factor and performs A / D (analog-digital) conversion to generate individual sound collection sound data. To do. The sound collection control unit 11 stores in advance a delay coefficient and an amplitude coefficient for each individual sound collection sound data for realizing different sound collection directivities. The sound collection control unit 11 performs filter processing based on these delay coefficient and amplitude coefficient for each set sound collection directivity for each individual sound collection sound data, thereby collecting sound collection directivity different from each other. Sound beam sound data is generated.

The sound collection control unit 11 compares the levels (sound levels) of the plurality of generated sound collection beam sound data, selects sound collection beam sound data that exceeds a preset sound detection threshold level, and outputs it to the echo canceller 12 To do.
The sound collection control unit 11 gives selection information for specifying the selected sound collection beam sound data to the main control unit 10.

  The echo canceller 12 includes an adaptive filter and a post processor. The adaptive filter generates pseudo-regression sound data based on the output audio data from the other party video conference apparatus output from the communication control unit 15 and gives the post-processor to the post-processor. The post processor includes an adder, and the adder subtracts the pseudo-regression sound data from the sound collection beam sound data output from the sound collection control unit 11 and performs echo cancellation to generate output sound data and control communication. To the unit 15. At this time, the post processor feeds back the echo cancellation result to the adaptive filter.

  The cameras CA <b> 1 to CA <b> 3 generate video data by imaging different imaging ranges as described above, and output the video data to the video control unit 13.

  The video control unit 13 generates composite video data using video data from each of the cameras CA1 to CA3 and preset image data for speaker identification. At this time, the video control unit 13 divides the entire screen of the composite video data into four equal partial image areas based on the center of the entire screen (see FIG. 5 (details will be described later)). The video control unit 13 allocates the videos of the cameras CA1 to CA3 to the three partial image areas, and allocates the remaining one partial image area to the speaker identification image data. The speaker identification image data includes speaker individual information such as a speaker name set in advance. Each piece of speaker individual information is stored in association with a camera that captures the corresponding speaker.

The video controller 13 forms a camera identification information display field for displaying camera identification marks corresponding to the cameras CA1 to CA3 in the partial image areas assigned to the cameras CA1 to CA3.
The video control unit 13 selects a camera identification mark corresponding to the speaker orientation and speaker individual information corresponding to the camera based on the speaker orientation camera information given from the main control unit 10. The video control unit 13 performs a setting to display the selected camera identification mark and speaker individual information in a manner different from other camera identification marks and speaker individual information. For example, the camera identification mark and speaker individual information of the selected speaker orientation are displayed with higher brightness than other camera identification marks and speaker individual information. This process is continuously performed at a predetermined frame rate set in advance. As a result, the speaker orientation changes as the speaker changes, and when the video data of the speaker speaking changes, the display mode of the camera identification mark and speaker individual information is also switched. Is done. The composite video data formed in this way is sequentially output to the communication control unit 15.

  The communication control unit 15 transmits the output audio data from the echo canceller 12 in association with the speaker orientation data, and transmits the composite video data from the video control unit 13.

  Thus, the other party (reception side) video conference device emits sound based on the output audio data and reproduces the composite image data, so that the conference person in the other party (reception side) conference room is present. Can listen to the speaker's remarks while watching the video of the entire conference room on the transmission side, and can easily identify the speaker by visually recognizing it. At this time, the voice of the collected sound beam is emitted, so that the speaker's speech can be emitted with a high S / N ratio, and the echo cancellation process further increases the S / N ratio. Sound can be emitted. As a result, it is possible to realize a video conference that is easy to understand and easy to use for the conference participants.

  When the communication control unit 15 receives the output voice data, the speaker orientation data, and the synthesized video data from the transmission side via the network 900, the communication control unit 15 outputs the output voice data to the sound emission control unit 14 via the echo canceller 12. To do. The communication control unit 15 outputs the speaker orientation data to the main control unit 10. The communication control unit 15 outputs the composite video data to the display device 20 separate from the video conference device 1. The display device 20 includes a liquid crystal display or the like, and reproduces and displays the composite video data input from the communication control unit 15.

  The sound emission control unit 14 is based on the output voice data generated by the other party from the communication control unit 15 and the sound source localization information from the main control unit 10 based on the speaker orientation data generated by the other party. Individual drive signals to be given to the speakers SP1 to SP14 are generated. More specifically, the sound emission control unit 14 distributes the output sound data to the speakers SP1 to SP14, and performs delay processing and amplitude processing based on the sound source localization information for each distributed sound data. Drive audio data is generated. The sound emission control unit 14 generates an individual drive signal by D / A (digital-analog) conversion of the generated individual drive audio data, and at a predetermined amplification factor based on the volume set by the operation unit 16. After amplification, it outputs to each speaker SP1-SP14.

  The speakers SP1 to SP14 emit sound based on the input individual drive signal. As a result, sound source localization based on the speaker orientation data is realized, and sound is emitted as if it was pronounced from the virtual speaker position.

  By performing such sound emission directivity control, together with the video display effect as described above, the conference can hear the speaker voice corresponding to the speaker orientation, and can make a video conference more realistic. Can be realized.

Next, a more specific usage example will be described with reference to the drawings.
FIG. 3 is a plan view showing an arrangement example of the video conference device 1 and an imaging range. FIG. 4 is a plan view showing an arrangement example of the video conference device 1 and the display 20.
As shown in FIG. 3, a conference table 400 is installed in the conference room, and conference participants 301 to 306 are seated so as to surround three sides of the conference table 400. Then, the video conference apparatus 1 is installed on the other side of the conference table 400. The video conference apparatus 1 is installed so that the front direction is the conference table 400 side. The video conference apparatus 1 is installed on the top surface of the display device 20, for example, as shown in FIG.

  The conference persons 301 and 302 are seated on the side facing the video conference apparatus 1 with respect to the conference table 400. The conference persons 303 and 304 are seated along the edge of the table 400 on the left end side (camera CA3 side) of the video conference apparatus 1, and the conference persons 305 and 306 are the conference tables on the right end side (camera CA2 side). Sitting along the 400 edges.

  FIG. 5 is a diagram illustrating an example of video data of each camera CA1 to CA3 in the situation of FIG. 3, (A) is an example of video data of camera CA1, (B) is an example of video data of camera CA2, and (C) is an example. An example of video data of the camera CA3 is shown.

  When the conference participants 301 to 306 are seated and the video conference apparatus 1 is activated as shown in FIG. 3, the camera CA1 captures an area including the conference participants 301 and 302 as shown in FIG. The video 311 of the conference 301 (hereinafter referred to as conference video 311), the video 312 of the conference 302 (hereinafter referred to as conference video 312), and the video 410 of the table 400 (hereinafter referred to as table video 410). Including video data 501 is generated. As shown in FIG. 5B, the camera CA <b> 2 captures an area including the conference participants 303 and 304, and a video 313 of the conference participant 303 (hereinafter referred to as conference participant video 313) and a video 314 of the conference participant 304. Video data 502 including the conference video 314 (hereinafter referred to as the conference video 314) and the table video data 410 is generated. As shown in FIG. 5C, the camera CA3 captures an area including the conference participants 305 and 306, and an image 315 of the conference participant 305 (hereinafter referred to as a conference participant video 315) and an image 316 of the conference participant 306 are obtained. Video data 503 including the conference video 316 (hereinafter referred to as conference video 316) and table video data 410 is generated.

The video control unit 13 forms composite video data 500 based on these video data 501 to 503.
6A and 6B are diagrams showing examples of composite video data. FIG. 6A shows a state where the conference person 301 speaks, and FIG. 6B shows a state where the conference person 303 speaks.
The video control unit 13 divides the display area of the composite video data 500 into four partial video areas 511 to 514 with reference to the center of the display area, and the video data 501 to 503 and the speaker identification image. Allocate data.
More specifically, the video control unit 13 assigns the video data 501 to the partial video area 511 corresponding to the second quadrant with the reference point as the center of the XY coordinate system, and the video data to the partial video area 512 corresponding to the first quadrant. 502 is allocated, and video data 503 is allocated to the partial video area 513 corresponding to the third quadrant. The video control unit 13 allocates speaker identification image data including pre-stored speaker individual information 541 to 546 to the partial video area 514 corresponding to the fourth quadrant.

  The video control unit 13 further assigns camera identification marks 521 to 523 to the partial video areas 511 to 513 to which the video data 501 to 503 are assigned. The camera identification mark is associated with each of the cameras CA1 to CA3, the “C” mark is set for the camera CA1, the “R” mark is set for the camera CA2, and the camera CA3 is set. An “L” mark is set. These camera identification marks 521 to 523 are arranged so as to contact each other around the reference point.

  In such a basic display mode, the video control unit 13 displays, based on the speaker orientation camera information, a camera identification mark for the camera that captures the speaker and speaker individual information associated with the camera with high brightness. Set the display. For example, when it is detected that the conference person 301 is a speaker, as shown in FIG. 6A, the camera identification mark 521 in the partial video area 511 of the video data 501 including the conference person video 311 is lit in high brightness yellow. The other camera identification marks 522 and 523 are set to be displayed in a low-luminance gray lighting state. Furthermore, the individual speaker information 541 and 542 corresponding to the camera CA1 in the partial video area 514 is displayed so as to be in a high luminance yellow lighting state, and the other speaker individual information 543 to 546 is in a low luminance gray lighting state. Set to display as follows. Then, when it is detected that the speaker has been switched to the conference 313, as shown in FIG. 6B, the camera identification mark 522 in the partial video area 512 of the video data 502 including the conference video 313 is displayed in a high-intensity yellow color. A setting is made so that the camera is displayed in a lighting state and the other camera identification marks 521 and 523 are displayed in a low-luminance gray lighting state. Further, the individual speaker information 543, 544 corresponding to the camera CA2 in the partial video area 514 is displayed so as to be in a high luminance yellow lighting state, and the other individual speaker information 541, 542, 545, 546 is displayed in a low luminance gray. Set the display so that it is lit.

  By playing the composite video data with such display settings, the playback-side conference can easily identify the video including the speaker and easily check the individual information of the speaker. Can do. In the case of FIG. 6, since two conference persons are captured in one video, two pieces of speaker individual information are displayed in a different manner from the other speaker individual information. If one conference person is imaged, the individual information of the speaker can be confirmed more reliably.

  In the above description, on the basis of the speaker orientation camera information, all the speaker individual information associated with the camera that captured the area including the speaker orientation is displayed in a display mode different from the other speaker individual information. Set. However, in the video conference apparatus according to the present embodiment, it is possible to detect the azimuth in units of conferences, not in units of imaging areas of the cameras CA1 to CA3. Therefore, the main control unit 10 may generate more detailed speaker direction information together with the speaker direction camera information and provide the information to the video control unit 13.

  In this case, the video controller 13 stores speaker orientation information and speaker individual information in association with each other in advance. When the video control unit 13 receives the speaker orientation camera information and the speaker orientation information from the main control unit 10, the video control unit 13 executes the process related to the camera identification mark and the process related to the individual speaker information based on the speaker orientation camera information. At the same time, speaker individual information related to the true speaker orientation is acquired from the selected plurality of speaker individual information. Then, the video control unit 13 sets the individual speaker information related to the true speaker direction so as to have another display mode.

FIG. 7 is a diagram illustrating an example of a display image when the display mode is set for each speaker. FIG. 7 shows a case where the conference person 301 speaks in the same manner as the situation of FIG. 3 and FIG.
As shown in FIG. 7, the display modes of the partial video area 511 including the video data 501, the partial video area 512 including the video data 502, and the partial video area 513 including the video data 503 are the same as those in FIG. However, in the partial video area 514, the speaker individual information 541 and 542 corresponding to the camera CA1 is displayed so as to be turned on, and the other speaker individual information 543 to 546 is displayed so as to be turned off. Do. Furthermore, the speaker individual information 541 corresponding to the speaker 301 who is the speaker is turned on or blinked with higher brightness than the speaker individual information 542 corresponding to the speaker 302 who is not the speaker. Set up.

  By setting it as such a display mode, a speaker can be identified reliably at a glance with the above-mentioned effect.

  In the above description, only a display example of the partial video area 514 has been shown. However, the display contents can be set by setting on the Web screen as shown in FIG.

FIG. 8 is a diagram showing a camera and conference information setting input screen.
When setting the Web screen, the user connects a device such as a personal computer to the network 900 and accesses the video conference apparatus 1. Thereafter, the user performs remote operation input, and displays a Web setting screen 600 as shown in FIG. 8 on the display screen of the personal computer. On the Web display screen 600, a “default” selection field 601 and a “participant information” selection field 602 for selecting a setting method are set. The user selects one of the setting methods by using an operator such as a mouse to select a setting method. The “participant information” selection column 602 includes a content display column 621 indicating the content, and the content display column 621 includes a display content input column 6211 and a camera assignment setting column 6212.

  In the display content input field 6211, personal identification information such as the name of a conference person as shown in FIG. 8 is input. Although FIG. 8 shows an example in which a name is input, the input content is not particularly specified as long as it is information that can be identified by an individual. The camera assignment setting field 6212 is a field for selecting assignment of the cameras CA1 (C), CA2 (R), and CA3 (L) for each input unit of the individual identification information in the display content input field 6211. If it is input, one is selected. The display mode of the partial video area 514 is set based on this selection content and the above-described speaker orientation camera information. That is, personal identification information in which the same camera as the speaker orientation camera information is selected is displayed in a display mode different from other personal identification information. Further, when the personal identification information is not input, “Not use” is selected in the corresponding camera assignment setting column 6212. Thereby, only the input personal identification information is displayed.

  After such a setting is made, by selecting the setting button field 631, the input content is determined and reflected on the video control unit 13. Furthermore, if the reset button column 632 is selected, the input content is reset.

  By performing such setting operation input, a display screen as shown in FIG. 6 can be obtained.

  When the “default” selection field 601 is selected, an image set in advance at the time of factory shipment is displayed in the partial video area 514. For example, the name and time of the video conference device, the conference elapsed time, and the like are displayed. If “not used” is selected, a video area corresponding to “not used” in the partial video area 514 becomes empty, and the video area may be assigned to a text information display area by external input. . In other words, the above-described personal computer or the like may be controlled so that a text input field is provided on the Web screen and the content input in the text input field is displayed. Accordingly, simple text information such as matters determined during the conference by the conference participants can be input, and the conference participants in each conference room can share this information.

  Moreover, although the example which installed all the functions in the video conference apparatus 1 main body was shown in the above-mentioned description, the communication control part 15 and a part of function of the main control part 10 accompanying this are a sound emission sound collection function, It may be separate from the imaging function. Further, the number of speakers, the number of microphones, and the number of cameras described above are examples, and may be set as appropriate according to the specifications of the apparatus. That is, the number of speakers is set according to a desired sound emission characteristic, and the number of microphones is set according to a desired sound collection characteristic. The number of cameras may be more or less than three depending on the desired imaging area, the presence or absence of an imaging area dividing function, and the like. However, the number of cameras is more preferably three. This is because the split screen, which is a feature of the present invention, is a four-screen split based on the screen center in consideration of user visibility, and one of the four split areas is used for displaying character information. A maximum of three screens are set for displaying the conference video. Therefore, if the number of cameras is four or more, images cannot be displayed on three screens at the same time. For example, when images including speakers are not displayed, images from four or more cameras can be displayed automatically. It must be allocated to three divided areas. For this reason, more complicated processing is required. Furthermore, when the number of cameras is larger than the number of divided areas, the entire conference room cannot be continuously imaged. Therefore, in the present invention, it is desirable that the number of cameras is three.

  In the above description, an example in which the sound collection function, the imaging function, and the sound emission function are installed in one housing is shown. However, the sound collection function and the imaging function are integrated, and there is no sound emission function. You may make it install a thing and a sound emission function separately.

  Further, in the above description, an example in which high-intensity yellow lighting and low-intensity gray lighting are used as a display for identifying a speaker has been shown. However, these lighting modes are merely examples, and the presence or absence of a speaker is clearly identified. If it is a lighting mode that can be set, it can be set as appropriate including lighting.

It is an external appearance perspective view of the video conference apparatus of this invention. It is the figure which showed the structure of the functional block of the video conference apparatus 1 of this invention, and the connection relationship with the exterior. 3 is a plan view illustrating an arrangement example and an imaging range of the video conference device 1. FIG. 3 is a plan view illustrating an arrangement example of a video conference device 1 and a display device 20. FIG. It is a figure which shows the image data example of each camera CA1-CA3 in the condition of FIG. It is a figure which shows the example of synthetic | combination video data. It is a figure which shows the example of the display image at the time of performing the display mode setting per speaker. It is a figure which shows the setting input screen of a camera and meeting information.

Explanation of symbols

  1-video conference device, 10-main control unit, 11-sound collection control unit, 12-echo canceller, 13-video control unit, 14-sound emission control unit, 15-communication control unit, 16-operation unit, 20- Display, MC1 to MC16-Microphone, SP1 to SP14-Speaker, CA1 to CA3-Camera, 301 to 306-Conference, 311 to 316-Conference video, 400-Conference table, 410-Table video, 500-Composite video Data, 501-503 video data, 511-514 partial video area, 521-523 camera identification mark, 541-546 individual speaker information

Claims (4)

A plurality of microphones each collecting sound and generating collected sound data;
Based on the collected sound data of the plurality of microphones, sound collection beam sound data generating means for generating a plurality of sound collection beam sound data having sound collection directivities centering on different directions, respectively,
Imaging means for generating a plurality of video data each consisting of different imaging areas;
A composite video data generating unit that generates a composite video data by combining a plurality of video data from the imaging unit;
Communication control means for outputting the collected sound beam audio data and the synthesized video data;
A video conferencing apparatus comprising:
A speaker orientation detecting means for detecting a speaker orientation based on a level of the plurality of collected beam sound data;
The synthesized video data generating means includes
Assigning each of the plurality of video data to each area obtained by dividing the composite video data into a screen;
Allocate speaker display data in which information indicating a speaker to be imaged for each of a plurality of video data output from the imaging means is allocated to an area of the composite video data to which the plurality of video data is not allocated. ,
Apply the imaging area specific mark corresponding to each video data individually in the screen division area of each video data,
The speaker display data and the imaging region specifying mark corresponding to the speaker orientation detected by the speaker orientation detecting means are displayed in a display mode different from the speaker display data and the imaging region specifying mark other than the speaker orientation. ,
Video conferencing equipment. The speaker display data comprises a plurality of individual speaker display data,
The synthesized video data generating means includes
Based on the speaker orientation, in the speaker display data, the individual speaker display data corresponding to the speaker orientation is further displayed in a different manner.
The video conference apparatus according to claim 1. The sound collecting beam sound data generating means selects and outputs sound collecting beam sound data corresponding to the speaker orientation.
The video conference apparatus according to claim 1 or 2. The communication control means has a function of receiving sound emission sound data corresponding to the sound collection beam sound data generated by the destination device and a speaker orientation associated with the sound emission sound data. ,
A plurality of speakers emitting sound based on the input speaker driving signal;
A sound emission directivity is set based on the sound emission sound data received by the communication control means and a speaker orientation associated with the sound emission sound data, and the plurality of sound emission directivities are set according to the sound emission directivity. Sound emission control means for generating the speaker drive signal for each of the speakers;
The video conference apparatus according to claim 1, comprising:

Images