JP2012027339A - Karaoke device - Google Patents

Abstract

An image in which a singer's figure is continuously captured with a single camera is reliably generated.
A karaoke apparatus 10 includes an apparatus main body 100, a microphone 300 that generates marker light, and an all-around camera 400 that captures a predetermined range of view including the microphone 300 and a singer and generates video data. ing. The apparatus main body 100 acquires sound data 106 and the like, a display unit 109 that displays background video data and lyrics data, and video data in a predetermined range obtained by photographing with the all-around camera 400, and is included in the video data. A control unit 101 that identifies the position of the microphone 300 based on the marker light, cuts out partial video data including the position of the microphone 300 from the video data in the karaoke room KR, and corrects the partial video data to obtain an output video. And have.
[Selection] Figure 5

Description

  The present invention relates to a karaoke apparatus that provides a reproduction service for karaoke performance songs.

  In recent years, in the karaoke apparatus, in addition to the reproduction of the karaoke performance music, various services accompanying this have been proposed. For example, a karaoke apparatus that shoots a singer's appearance with a single video camera and displays it on a display has been proposed in the past (for example, see Patent Document 1). This conventional karaoke device receives one video camera, a servo head for freely changing the orientation of the video camera within a predetermined range, and a radio beacon signal from a wireless microphone of the singer. Drive control means for driving the servo head so that the direction of the video camera is aligned with the position of the source of the radio signal.

JP-A-10-240276

  However, in the case of the technique of tracking and shooting a singer by rotating a video camera with a narrow visual field range based on a radio beacon signal from a microphone as in the above prior art, the singer moves his seat while singing or is fast In the case of movements, the singer's figure could momentarily frame out. In order to prevent this, the singer needs to restrict his movement and movement so as not to be out of frame, which is inconvenient.

  The objective of this invention is providing the karaoke apparatus which can produce | generate reliably the image | video which image | photographed the figure of the singer seamlessly with one camera.

  In order to achieve the above object, the first invention is a karaoke apparatus that provides a reproduction service of karaoke performance music using music data and video data, the music reproduction means for reproducing the music data, As the music data is played back by the music playback means, the display means capable of displaying the video data, a microphone possessed by the singer and used to input a voice signal of karaoke singing by the singer, and A sign signal generating means for generating a sign signal and a predetermined field of view including the microphone and the singer provided in the microphone, and the sign signal generating means including the sign signal generated from the sign signal generating means. One moving picture camera for generating video data, and the predetermined range of video data generated by the one moving picture camera. Video extraction means for specifying the position of the microphone based on the marker signal included in the video signal and cutting out the partial video data including the specified position from the video data in the predetermined range; and the partial video cut out by the video extraction means And display control means for performing a predetermined correction process on the data and displaying the corrected image on the display means.

  In the karaoke apparatus of the first invention of the present application, at the time of karaoke performance, the music data is reproduced by the music reproducing means and the video data is displayed by the display means, and the singer sings with the microphone in accordance with the reproduction and display. Do. In addition, the karaoke apparatus is provided with one moving image shooting camera. At the time of singing, the one video camera captures a predetermined field of view including the singer and the microphone, and generates video data of the predetermined range. At this time, for example, by using a video camera with a wide field of view (for example, a camera equipped with a fisheye lens and capable of shooting 360 ° around the circumference), one singer moves while singing or a plurality of singers sing simultaneously. Even if the singers are sequentially changed, the singers can always be included in the video data generated by the video camera.

  Here, the microphone of the first invention of the present application is provided with a sign signal generating means for generating a sign signal. Therefore, a sign signal corresponding to the position of the microphone possessed by the singer is always recorded together with the appearance of the singer in the generated video data in the predetermined range. Accordingly, in response to this, the video extraction means specifies the position of the microphone using the marker signal included in the video data, and cuts out the partial video data including the specified position from the video data in the predetermined range. Since the microphone is possessed by the singer, the display control means performs a predetermined correction process on the clipped partial video data, so that the video after the correction process including the singer can be displayed on the display means. it can.

  As described above, in the first invention of the present application, a wide field of view is always photographed by one video camera, and video data including a singer is cut out from the video data by using a sign signal as a clue. By correcting further, the correct video of the singer is obtained. As a result, it is possible to prevent the appearance of the singer from being momentarily out of the frame, as in the case where the video camera with a narrow visual field range is rotated based on the radio beacon signal from the microphone to track and photograph the singer. That is, according to the first invention of the present application, it is possible to reliably generate an image in which a singer's figure is continuously captured while using one camera. As a result, even if the user wants to take a picture of himself / herself during singing, the singer can sing in a wide range without restrictions on the place, or can sing while moving freely.

  According to a second aspect of the present invention, in the first aspect of the present invention, there is provided audio determination means for determining whether or not an input level of an audio signal input from the microphone is equal to or higher than a predetermined value. Is determined to be greater than or equal to a predetermined value, the marker signal from the marker signal generating means specifies the position of the microphone included in the video data of the predetermined range, and the specified position from the video data of the predetermined range It is characterized by cutting out partial video data including

  Thereby, the position of the microphone whose input level of the audio signal is less than the predetermined value is excluded from the target for cutting out the partial video data. Therefore, it is possible to reliably cut out only the video when the singer who possesses the microphone sings as partial video data. In addition, for example, when a plurality of persons each possessing a plurality of microphones are switched in order and singing or singing alternately, only the image of the person singing at that time among the plurality of persons is surely obtained. Can be cut out.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the plurality of indicator signal generating means provided in each of the plurality of microphones generates the indicator signals of different colors, and the video extracting means A feature of distinguishing and specifying the position of each of the plurality of microphones based on the color of the marker signal included in the video data of the range, and cutting out the partial video data at each specified position from the video data of the predetermined range, To do.

  Thereby, when a plurality of singers sing with different microphones, the respective partial video data can be cut out while specifying each singer for each color of the microphone. In addition, by cutting out all the videos of a plurality of singers sung at the same time as partial video data, it is possible to simultaneously convert a plurality of singers into video data with one camera (identifying the singer). .

  According to a fourth invention, in any one of the first to third inventions, when the music data is played back by the music playing means, the voice signal of the karaoke song input from the microphone is used for each microphone. Each of the scoring means for scoring and the video extracting means specify the position of the specific microphone whose scoring result by the scoring means is greater than or equal to a predetermined value using the marker signal, and the partial video at the specified position Data is cut out from the predetermined range of video data.

  Thereby, only when a singer sings well, the figure can be cut out as partial video data, and entertainment can be improved. Also, for example, when multiple singers each possessing multiple microphones sang, only a partial image of those multiple singers who are at or above the level with a scoring result or who has the best scoring result It can also be cut out as data. In this case, entertainment can be further improved by a game sensation enjoyed by a plurality of people.

  According to the present invention, it is possible to reliably generate a video in which the appearance of a singer is seamlessly captured with a single camera.

It is a figure which represents roughly the karaoke room in which the karaoke apparatus of one embodiment of this invention was installed. It is a figure which represents typically the image acquired when the inside of a karaoke room is image | photographed with the all-around camera. It is a side view showing the external appearance of a microphone. It is a functional block diagram showing the whole structure of the karaoke system provided with the karaoke apparatus. It is explanatory drawing showing the process of obtaining the moving image data of a singer by image-processing the image | video input from the perimeter camera. It is a flowchart showing the detail of the process sequence performed by the control part of an apparatus main body. It is a figure showing the example of a display of the karaoke moving image uploaded to the host server. It is explanatory drawing showing an example of the area | region where one singer moves during singing. It is explanatory drawing showing the video data in the comparative example of this invention. It is a conceptual explanatory drawing showing the acquisition principle of the video data in one Embodiment of this invention. It is a figure showing the video data in one Embodiment of this invention. It is a functional block diagram showing the whole structure of the karaoke system provided with the karaoke apparatus of the modification provided with the some microphone from which the color of the marker light to generate differs. It is a flowchart showing the detail of the process sequence performed by the control part of an apparatus main body. It is explanatory drawing showing an example of the area where the green marker light microphone exists, and the area where the red marker light microphone exists. It is a graph showing an example of the time transition of the audio | voice input level of a green marker optical microphone and a red marker optical microphone. It is a flowchart showing the detail of the process sequence performed by the control part of an apparatus main body in the modification which cuts out considering the scoring result of a karaoke song.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram schematically showing a karaoke room in which the karaoke apparatus of the present embodiment is installed.

  In FIG. 1, a karaoke apparatus 10 is installed in a karaoke room KR such as a karaoke store. The karaoke device 10 is a device that provides a karaoke performance reproduction service using Musical Instrument Digital Interface (MIDI) data and video data as music data. In the one shown in FIG. 1, users A to C perform karaoke singing. The karaoke apparatus 10 includes an apparatus main body 100 called a commander, a remote controller 200, a microphone (hereinafter abbreviated as a microphone) 300, and a single all-around camera 400 (moving picture camera). The apparatus main body 100, the remote controller 200, and the microphone 300 will be described in detail later.

  In this example, the all-around camera 400 is provided downward in the center of the ceiling of the karaoke room KR, for example. The all-around camera 400 captures a fixed range of visual field including the microphone 300 and the singers A to C, and generates video data of the range. Specifically, the all-around camera 400 has a fisheye lens, and obtains video data in which a hemispherical field of 360 degrees on the left and right and 90 degrees on the top and bottom is contained in one sheet. Due to the function of the fisheye lens, the image in the field of view of the omnidirectional camera 400 has a property that a closer object appears in the center of the circle and a farther object appears in the periphery of the circle. In addition, as a price for obtaining a wide viewing angle with a fisheye lens, all objects are distorted in a fan shape.

  Therefore, when the entire inside of the karaoke room KR is photographed by the all-around camera 400, an overall fan-shaped image is obtained (schematically shown in FIG. 2).

  FIG. 3 is a diagram illustrating the appearance of the microphone 300. In FIG. 3, the microphone 300 converts the voice of karaoke singing by the user into a voice signal and inputs it.

  The microphone 300 has a microphone housing 302 in which a microphone element 301 is built. A power switch 303 is provided on the top of the microphone housing 302. In the lower part of the microphone housing 302, a light emitting diode (LED) 304 that emits marker light as an optical marker signal of a predetermined color (in this example, green) and green marker light emitted from the LED 304 are uniformly diffused. A translucent light diffusion sphere 305 is provided.

  FIG. 4 is a functional block diagram showing the overall configuration of the karaoke system including the karaoke apparatus 10 described above.

  In FIG. 4, the karaoke system 1 includes the karaoke apparatus 10 installed in the karaoke room KR and a host server 20. The karaoke apparatus 10 and the host server 20 are connected to each other so as to be able to transmit and receive information via a network NW such as a communication network.

  The karaoke apparatus 10 includes the apparatus main body 100, the remote controller 200, the microphone 300, and the all-around camera 400. The apparatus main body 100 and the remote controller 200 are connected to each other so as to be able to transmit / receive information to / from each other via a network such as a wireless or wired LAN. The apparatus main body 100 and the microphone 300 are connected by a wireless line or a wired line.

  The apparatus main body 100 includes a control unit 101, a mass storage device 103, an operation unit 104, a reception unit 105, a sound source 106, a sound control unit 107, a speaker 108, a display unit 109, and a communication control unit 110. And have.

  The control unit 101 includes a CPU (not shown) and a memory such as a RAM and a ROM. The control unit 101 executes various programs stored in advance in the ROM or the large-capacity storage device 103 while using the temporary storage function of the RAM. Thereby, the entire apparatus main body 100 is controlled. In particular, the control unit 101 performs predetermined image processing on the captured image in the karaoke room KR obtained by the omnidirectional camera 400, and includes a plurality of images of a singer who has the microphone 300 (a plurality of videos that can form a moving image). (The same applies below), and the image is stored in the mass storage device 103 and displayed on the display unit 109 (details will be described later).

  The mass storage device 103 is composed of, for example, a hard disk drive (HDD). The mass storage device 103 stores various information such as MIDI data, background video data, and lyrics data. Also, the large-capacity storage device 103 sequentially stores moving image data when the user sings.

  The operation unit 104 includes, for example, a plurality of keys and switches. The user can perform various operations such as a reservation operation for a karaoke performance using the operation unit 104 or an operation unit 204 of the remote controller 200 described later.

  The receiving unit 105 receives the singer's voice signal output from the microphone 300.

  The sound source 106 reproduces the MIDI data read from the mass storage device 103 by the control unit 101 and outputs it to the audio control unit 107. The audio control unit 107 amplifies the MIDI data output from the sound source 106 and the audio signal input from the microphone 300 via the receiving unit 105 and outputs the amplified signal to the speaker 108. The speaker 108 outputs the MIDI data and audio signal output from the audio control unit 107 as audio.

  Hereinafter, the sound source 106, the audio output unit 107, and the speaker 108 are appropriately omitted and referred to as “sound source 106 etc.”. The sound source 106 and the like constitute music reproducing means for reproducing music data.

  The display unit 109 is composed of a liquid crystal display, for example, and functions as display means for displaying various videos. In particular, the display unit 109 synchronizes with the reproduction of the MIDI data by the sound source 106 or the like, in other words, the background video read from the mass storage device 103 as the MIDI data is reproduced by the sound source 106 or the like. Data and telops corresponding to the lyrics data can be displayed.

  The communication control unit 110 controls information communication with the remote controller 200 and the host server 20.

  The remote controller 200 is an operation terminal for a user to perform various operations such as a reservation operation for a karaoke performance song. The remote controller 200 includes a control unit 201, a storage device 203, an operation unit 204, a display unit 209, and a communication control unit 210.

  The control unit 201 includes a memory such as a CPU, RAM, and ROM (not shown). The control unit 201 executes various programs stored in advance in the ROM or the storage device 203 while using the temporary storage function of the RAM. As a result, the entire remote controller 200 is controlled.

  The storage device 203 is composed of, for example, a nonvolatile memory and stores various types of information. The operation unit 204 is composed of, for example, a plurality of keys and switches. Using the operation unit 204 or the operation unit 104 of the karaoke apparatus 100, the user can perform various operations such as a reservation operation for karaoke performance songs. The display unit 209 is composed of, for example, a liquid crystal display and performs various displays.

  The communication control unit 210 controls information communication with the apparatus main body 100 and the host server 20.

  The host server 20 can upload the moving image data of the user's singing as a compressed moving image file (details will be described later). The moving image data uploaded to the host server 20 can be viewed from a specific user terminal on a predetermined Web page (see also FIG. 7 described later).

  Here, the greatest feature of this embodiment is that the position of the microphone 300 is specified based on the marker light included in the video image in the karaoke room KR obtained by the all-around camera 400, and includes the position of the microphone 300. This is to cut out the partial video and acquire the video (karaoke post video) data of the singer with the microphone 300.

  By the way, since the video signal obtained by imaging with the all-around camera 400 is significantly different from the normal visual field of human beings, it cannot be used as it is for a karaoke posted moving image. For this reason, it is necessary to perform a predetermined process on the video signal obtained by imaging with the omnidirectional camera 400.

  FIGS. 5A to 5F are explanatory diagrams illustrating a process of obtaining video data of a singer by performing image processing on the video input from the all-around camera 400. This process is executed by the control unit 101 of the apparatus main body 100.

  As shown in FIG. 5A, first, an image in the karaoke room KR acquired by the all-around camera 400 is input. Here, only the images of the users A to C and the microphone 300 are shown, and the images of the table, the apparatus main body 100, and the like are omitted.

  Thereafter, as shown in FIG. 5B, color filter processing is performed to remove components other than a component of a predetermined color (in this example, green) in the video input from the all-around camera 400. Specifically, only the green component is extracted from the video input from the all-around camera 400 through a color filter (here, a green filter). The color filter is a pixel data file obtained by CPU calculation processing that passes only the G values of RGB or passes only the pixel values of YUV UV within a certain range.

  Thereafter, as shown in FIG. 5C, the image data that has been subjected to the color filter processing is passed through a luminance filter, and only pixel values that have a luminance of a certain value or higher are passed, thereby binarizing the image data. I do. Thereby, only a pixel “close to pure green and having a certain level of brightness” in the image indicates “1”, and other pixels indicate “0”.

  Thereafter, as shown in FIG. 5D, area determination is performed on the video input from the all-round camera 400. Specifically, according to a map defined in the shape of a spider web in advance, the angle direction (corresponding to left and right in human vision) is divided into 8 (A to H) and the distance direction (in human visual depth). All the pixel values of the bitmap are added for each area for each area that has been divided into a total of 28 (three) (1-3) or four (1-4). The value with the largest added value (area G2 in the figure) detects the marker light and is determined to be the area (direction) where the singer to be photographed is. In addition, the boundary part of adjacent areas among areas A1-H4 overlaps partially, for example.

  Thereafter, as shown in FIG. 5E, the shape of the image is corrected by fan scan for the area selected in the process shown in FIG. Specifically, a process of rearranging the pixels in the area according to the scanning line shown in FIG. As a result, the fan-shaped diameter direction is deformed and corrected to a Y-axis rectangular shape and the circumferential direction is an X-axis rectangular shape. Here, the scanning line close to the central part is short and the scanning line close to the circumferential part is long, but it is supplemented with pixels of the same value and enlarged, or the pixels are omitted and reduced to obtain line data of a fixed length. . A scanning line coordinate pattern is prepared in advance for each of the areas A1 to H4.

  When the conversion process is completed for all the scanning lines, a final output image (an image of area G2 in the drawing) as shown in FIG. 5F is obtained. This image is obtained by cutting out only the peripheral region of the marker light captured by the all-round camera 400 and converting and correcting the circular viewing angle of the curved fish-eye lens into a normal rectangular viewing angle. The result is equivalent to pointing a normal camera to a singer with 300.

  Since these processes are performed on each image constituting the moving image, the process is performed at a rate of 30 frames per second. However, for example, the process is performed at a frequency of 1 frame per 10 frames because of the calculation capability. , Thinning may be performed.

  FIG. 6 is a flowchart showing details of a processing procedure executed by the control unit 101.

  In FIG. 6, when the reproduction of the MIDI data corresponding to the karaoke performance music is started, this flow is started. That is, the background video data and the lyrics data are displayed on the display unit 109 in synchronization with the reproduction of the MIDI data by the sound source 106 or the like. Then, the singing of the karaoke performance music by the singer is performed, and the voice of the karaoke singing is input from the microphone 300.

  First, in step S10, video data in the karaoke room KR taken by the all-around camera 400 is acquired. Then, in step S15, it is determined whether the marker light of the microphone 300 is detected based on the video data in the karaoke room KR acquired in step S10. When the marker light of the microphone 300 is detected, the determination of step S15 is satisfied and the process proceeds to step S20. When the marker light of the microphone 300 is not detected, the determination of step S15 is not satisfied and the process returns to step S10. Repeat the procedure.

  In step S20, it is determined whether or not the input level of the audio signal of the microphone 300 input via the receiving unit 105 and the audio control unit 107 is equal to or higher than a predetermined value. When the input level of the audio signal of the microphone 300 is equal to or higher than the predetermined value, the determination in step S20 is satisfied and the process proceeds to step S25. On the other hand, when the input level of the audio signal of the microphone 300 is not equal to or higher than the predetermined value, the determination in step S20 is not satisfied, the process returns to step S10, and the same procedure is repeated.

  In step S25, the position of the microphone 300 is specified based on the marker light included in the video data in the karaoke room KR. The processing in step S25 corresponds to the image processing shown in FIGS. 5 (a) to 5 (c). In step S30, partial video data including the position of the microphone 300 is cut out from the video data in the karaoke room KR. The process in step S30 corresponds to the area determination process shown in FIG.

  Thereafter, in step S35, correction processing of the partial video data including the position of the microphone 300 is performed, and an output video in which the figure of the singer holding the microphone 300 is captured is obtained. The process of step S35 corresponds to the shape correction of the image shown in FIGS. 5 (e) and 5 (f). In step S <b> 40, the output video after the correction process is displayed in a partial area of the display unit 109 and stored in the large capacity storage device 103.

  Thereafter, in step S45, it is determined whether or not the reproduction of the MIDI data corresponding to the karaoke performance song has been completed. When the reproduction of the MIDI data corresponding to the karaoke performance song is completed, the determination at step S45 is satisfied, and the routine goes to step S50. On the other hand, when the reproduction of the MIDI data corresponding to the karaoke performance song is not finished, the determination in step S45 is not satisfied, and the process returns to step S10 and the same procedure is repeated. Thus, steps S10 to S40 are repeated while the reproduction of the karaoke performance song is not completed, and the corrected video is sequentially stored in the large capacity storage device 103 in step S40 every time step S40 is passed.

  In step S <b> 50, the moving image posting instruction operation screen is displayed in a partial area of the display unit 109. Thereafter, in step S55, it is determined whether posting of a karaoke video is instructed by an operator (for example, any of users A to C) on the video posting instruction operation screen of the display unit 109. When the posting of the karaoke video is instructed, the determination at step S55 is satisfied, and the process proceeds to step S60. When the posting of the karaoke video is not instructed, the determination at step S55 is not satisfied and the flow ends.

  In step S60, the moving image data using the plurality of output videos after the correction processing sequentially stored in the large-capacity storage device 103 in step S40 at the time of the above repetition is uploaded to the host server 20, and this flow is finished.

  Note that, in the above, step S10, step S15, step S25, and step S30 shown in FIG. 6 function as the video extraction unit described in each claim, and step S35 and step S40 function as the display control unit. Moreover, step S20 shown in FIG. 6 functions as a voice determination unit.

  A display example of a karaoke video uploaded to the host server 20 is shown in FIG. In the display example shown in FIG. 7, in addition to the image sung by a singer (for example, any of users A to C. The user A singing according to the examples of FIGS. 1 and 2), karaoke The name of the performance song, the singing date and time, the number of times the karaoke video is played, etc.) are displayed. In this example, after the browsing of a specific user on the predetermined Web page described above, the browsing user's evaluation on the karaoke video (“delicious”, “interesting”, “cute”, “crying” is also included) Filled in and displayed.

  As described above, in the present embodiment, during karaoke performance, MIDI data as music data is reproduced by the sound source 106 and the like, and background video data is displayed by the display unit 109, and the reproduction and display are performed accordingly. Then, a singer (user A in the above example) sings with the microphone 300. When singing by a singer, one all-around camera 400 captures a predetermined field of view of the karaoke room KR including the singer and the microphone 300, and generates video data thereof. At this time, by using the omnidirectional camera 400 with a wide field of view, even if the singer moves in the karaoke room KR while singing, the singer is always included in the video data generated by the omnidirectional camera 400. (It is not necessarily limited to the all-round camera, and any camera having a function capable of photographing a predetermined wide range in the karaoke room KR at a time is sufficient).

  Here, the video data obtained when one singer moves during singing will be described using a comparative example. Here, as shown in FIG. 8, a case where a singer standing and singing in the region P moves to the region R through the region Q and sings while sitting on a chair in the region R will be described. . The regions P to R correspond to, for example, any one of the areas A1 to H4 illustrated in FIG.

  FIGS. 9A to 9C show, as a comparative example of the present embodiment, a servo-type pan head driven and controlled on the basis of a radio beacon signal corresponding to the above-described conventional structure, and one video camera is rotated. The video data in the case where the singer is photographed. As described above, when the singer moves from the region P → the region Q → the region R during the singing, the video camera of this modified example tracks and shoots the singer because the visual field range is narrow. . That is, the video camera also shoots while sequentially performing the pan operation in the order of the region P, the region Q, and the region R so as to track the movement of the region P → the region Q → the region R of the singer.

  At this time, in the area P, the moving speed is zero immediately before the singer starts moving (or the moving speed is low immediately after the movement starts), so the singer in the area P is captured by the video camera. Video data is obtained (FIG. 9A). Similarly, in the region R, since the singer has finished moving and the moving speed is zero (or the moving speed is low just before the end of the movement), the singer is in the region R by the tracking function based on the rotation of the video camera. Video data that captures the appearance of the singer can be obtained (FIG. 9C).

  However, when the singer moves from the region P to the region Q to the region R relatively quickly, the tracking operation by the rotation of the video camera cannot catch up with the movement of the singer, and the video camera pans from the region P. When the region Q is photographed by performing an action (swinging motion), the singer has already moved to the region R side from the region Q, and the singer's figure is instantaneously framed out. (FIG. 9B). Therefore, if you want to shoot your own figure while singing, you need to restrict your movement and movement so that you don't get out of the frame, and you can't move freely during singing, which is inconvenient.

  FIGS. 10A to 10C are explanatory diagrams of video data when the area P, the area Q, and the area R are captured by the all-round camera 400 in the present embodiment. In the present embodiment, as described above, the entire area in the karaoke room KR (see the areas A1 to H4 in FIG. 5D) is always photographed by the fixed all-around camera 400. FIG. 10 (a) is a conceptual view of the case where the shape correction as described with reference to FIG. 5 (e) is performed, corresponding to the video data for each time of the region R out of all the regions. It is a figure showing a figure. FIG. 10B is a diagram showing video data corresponding to the time-dependent video data in the area Q, and FIG. 10C shows video data corresponding to the video data per time in the area P. It is a figure.

  In the present embodiment, as described above, the inside of the karaoke room KR is always photographed by the fixed all-around camera 400, and the marker light of the microphone 300 is used as a clue from the video data in the karaoke room KR. Identify the location. Then, by cutting out partial video data including the singer who possesses the microphone 300 and further performing a predetermined correction process on the partial video data, a correct video of the singer can be obtained. That is, as in the above comparative example, instead of rotating and tracking one video camera in the order of the region P → the region Q → the region R, in this embodiment, the region P, the region Q, In addition, the image data is acquired by always photographing the region R. In this embodiment, video data including the region P at the moment when the singer who moves in the order of the region P → the region Q → the region R is in the region P is cut out from the acquired video data ( 10C), the video data including the region Q at the moment when the singer is in the region Q is cut out (see the middle row in FIG. 10B), and the singer is in the region R at the moment. Video data including the region R is cut out (see the lower part of FIG. 10A). Accordingly, it is possible to reliably obtain video data in which the appearance of the singer that always captures the appearance of the singer moving in the order of the region P → the region Q → the region R and that the singer's appearance does not instantaneously frame out ( FIG. 11).

  As described above, according to the present embodiment, it is possible to reliably generate an image in which the appearance of the singer is seamlessly captured using a single all-around camera 400 with a wide field of view (see FIG. 11). As a result, even if it is a case where he / she wants to photograph himself / herself during singing, the singer can sing while moving freely in the karaoke room KR, and convenience can be improved.

  Further, in this embodiment, in particular, when the input level of the audio signal of the microphone 300 is equal to or higher than a predetermined value, the position of the microphone 300 is specified based on the marker light, and partial video data including a singer who owns the microphone 300 Cut out. Thereby, the microphone 300 whose audio signal input level is less than the predetermined value is excluded from the segmentation target of the partial video data. In the above example, in the microphone 300 possessed by the user A who is a singer, even if the switch of the microphone 300 is ON, if the user A stops singing, the microphone 300 is The included partial video data is not cut out. Therefore, only the video when the singer (user A) who possesses the microphone 300 is singing can be reliably cut out as partial video data.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and technical idea of the present invention. Hereinafter, such modifications will be described.

(1) When a plurality of microphones with different marker light colors are provided In the above embodiment, only one microphone 300 is provided. However, the present invention is not limited to this, and a plurality of microphones are provided. The color of the marker light emitted from may be different.

  FIG. 12 is a functional block diagram illustrating the overall configuration of a karaoke system including the karaoke apparatus of the present modification.

  In FIG. 12, a karaoke apparatus 10 according to this modification includes the apparatus main body 100, the remote controller 200, the all-around camera 400, and microphones 300A and 300B. The structure of the microphones 300A and 300B is the same as that of the microphone 300 described above. The LED 304 (refer to FIG. 3 described above) of the microphone 300A emits green marker light similarly to the microphone 300 described above, and the LED 304 (refer to FIG. 3 described above) of the microphone 300B emits red marker light.

  FIG. 13 is a flowchart showing details of a processing procedure executed by the control unit 101 in this modification. FIG. 13 corresponds to FIG. 6 described above. The same steps as those in FIG. 6 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In FIG. 13, after step S <b> 10 is executed, in step S <b> 15, it is determined whether at least one marker light of the microphones 300 </ b> A and 300 </ b> B has been detected based on the video data in the karaoke room KR. When at least one of the marker lights of the microphones 300A and 300B is detected, the determination is satisfied and the process proceeds to step S105. When neither of the marker lights of the microphones 300A and 300B is detected, the determination is not satisfied and the process proceeds to step S10. Return.

  In step S105, it is determined whether both microphones 300A and 300B are used, that is, whether the power switches of microphones 300A and 300B are both turned on. When both the microphones 300A and 300B are used, the determination in step S105 is satisfied, and the process proceeds to step S110. When only one of the microphones 300A and 300B is used, the determination in step S105 is not satisfied, and the process proceeds to step S115 described later.

  In step S110, a sound level input from a microphone 300A that emits green marker light (hereinafter referred to as a green marker light microphone) 300A is input from a microphone 300B that emits red marker light (hereinafter referred to as a red marker light microphone) 300B. It is determined whether the sound level is higher than the sound level to be played. When the sound level input from the green marker light microphone 300A is higher than the sound level input from the red marker light microphone 300B, the determination in step S110 is satisfied, and the process proceeds to step S120. When the sound level input from the green marker light microphone 300A is not higher than the sound level input from the red marker light microphone 300B, the determination in step S110 is not satisfied, and the process proceeds to step S135 described later.

  On the other hand, in step S115, it is determined whether or not only the green marker light microphone 300A is used. When only the green marker light microphone 300A is used, the determination is satisfied, and the process proceeds to step S120. When only the red marker light microphone 300B is used instead of the green marker light microphone 300A, the determination is not satisfied and the step is performed. The process moves to S135.

  In step S120, the position of the green marker light microphone 300A is specified based on the marker light of the green marker light microphone 300A included in the video data in the karaoke room KR. In step S125, partial video data including the position of the green marker light microphone 300A is cut out from the video data in the karaoke room KR. Thereafter, in step S130, the partial video data including the position of the green marker light microphone 300A is corrected, and the process proceeds to step S40 similar to the above. Note that the processing in steps S120 to S130 may be performed in the same manner as steps S25 to S35 shown in FIG.

  On the other hand, in step S135, the position of the red marker light microphone 300B is specified based on the marker light of the red marker light microphone 300B included in the video data in the karaoke room KR. In step S140, partial video data including the position of the red marker light microphone 300B is cut out from the video data in the karaoke room KR. Thereafter, in step S145, the partial video data including the position of the red marker light microphone 300B is corrected, and the process proceeds to step S40. Note that steps S135 to S145 need only be performed in the same manner as steps S25 to S35 shown in FIG.

  Since the processes in steps S45 to S60 after step S40 are the same as those in the above embodiment, the description thereof is omitted.

  The operation of this modification executed by the above flow will be described. For example, the case where the user A (see FIG. 1) has the green marker light microphone 300A and the user B (see FIG. 1) sings with the red marker light microphone 300B will be described as an example. In this case, as shown in FIG. 14, from the video data in the karaoke room KR, a partial image including the area G2 that is the position of the green marker light microphone 300A and a portion including the area C2 that is the position of the red marker light microphone 300B The video will be cut out.

  FIG. 15 is a graph showing an example of a time transition of the voice input level of the green marker optical microphone 300A possessed by the user A and the red marker optical microphone 300B possessed by the user B.

15, in this example, from time t ₀ to time t ₁ red marker light microphone 300B audio input level (in other words, for example, the sound level of the user B) is in other words sound input level of the green marker light microphone 300A ( For example, it is higher than the voice level of user A). As a result, image processing is performed with the red filter, and a partial video including the area C2 is cut out, and an image during the singing of the user B is obtained. On the other hand, from the time _{t 1} to time _{t 2,} the audio input level of the green marker light microphone 300A is higher than the audio input level of the red marker light microphone 300B. As a result, image processing is performed with the green filter, and a partial video including the area G2 is cut out, and an image of the user A who is singing is obtained. Since the time t ₂ to time t ₃ audio input level of the red marker light microphone 300B higher than the audio input level of green marker light microphone 300A, partial image including an area C2 performs image processing with the red filter is cut An image during the singing of the user B is obtained. Higher than the audio input level of the audio input level red marker light microphone 300B green marker light microphone 300A from time t ₃ to time t _4, partial video is cut out containing area G2 performs image processing with the green filter The image during the singing of the user A is obtained. Thus, in this modification, it will acquire so that it may switch to the animation of the singer who is singing out loud.

  In the flow shown in FIG. 13, the position of the microphone with the higher input audio level among the green marker light microphone 300 </ b> A and the red marker light microphone 300 </ b> B is specified, and the subsequent processing is sequentially executed. Similar to the flowchart shown in FIG. 6, when the sound level of the sound signal input from the microphones 300A and 300B is equal to or higher than a predetermined value, the positions of the microphones 300A and 300B are specified, and the subsequent processing is sequentially executed. May be. In this case, when the sound levels of the sound signals input from the microphones 300A and 300B are both high, images of the two singers singing with the microphones 300A and 300B can be obtained simultaneously. .

  According to this modification described above, the following effects can be obtained in addition to the same effects as those of the above embodiment. That is, when a plurality of users A to C having microphones 300A and 300B are sequentially switched and singing or singing alternately, singing at that time among the plurality of users A to C. Only the image of the user who is present can be cut out reliably. In addition, by extracting all the images of two singers sung at the same time as partial video data, the singers can be identified with a single all-around camera 400 and the images of the two singers can be converted into video data simultaneously. You can also

  In this modification, two microphones 300A and 300B are provided. However, three or more microphones may be provided and marker lights of different colors may be generated from each microphone.

(2) When cutting out considering the scoring result of karaoke singing In this modification, the voice of karaoke singing input from the microphones 300A and 300B is scored, and the positions of the microphones 300A and 300B are determined based on the scoring results. Cut out partial video data.

  FIG. 16 is a flowchart showing details of a processing procedure executed by the control unit 101 in this modification. FIG. 16 corresponds to FIG. 13 described above. The same steps as those in FIG. 13 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In FIG. 16, first, in step S150, scoring of the voice of the karaoke song input from the microphones 300A and 300B is started by a known appropriate method. This step S150 constitutes scoring means described in each claim. Thereafter, the process proceeds to step S10.

  Thereafter, in step S105, it is determined whether both of the microphones 300A, 300B are used. If both of the microphones 300A, 300B are used, the determination is satisfied and the process proceeds to step S155.

  In step S155, it is determined whether or not the karaoke song input from the green marker light microphone 300A has a higher score than the karaoke song input from the red marker light microphone 300B. When the score of the karaoke song input from the green marker light microphone 300A is higher than that of the karaoke song input from the red marker light microphone 300B, the determination in step S155 is satisfied and the process proceeds to step S120. When the score of the karaoke song input from the green marker light microphone 300A is not higher than the score of the karaoke song input from the red marker light microphone 300B, the determination in step S155 is not satisfied, and the process proceeds to step S135.

  Since the processes after step S120 and step S130 are the same as described above, the description thereof will be omitted. In the above, the position of the microphone with the higher score among the karaoke songs input from the green marker optical microphone 300A and the red marker optical microphone 300B is specified, and the subsequent processing is sequentially executed. Absent. That is, when the score of the karaoke song input from the microphones 300A and 300B is equal to or greater than a predetermined value, the positions of the microphones 300A and 300B may be specified, and the subsequent processing may be sequentially executed.

  In this modification, it is possible to cut out only the figure of a person who has a graded result or higher among a plurality of singers or who has the best graded result as partial video data. In this case, amusement can be improved by a game sensation enjoyed by a plurality of people.

  In this modification, two users sing using the microphones 300A and 300B, but the present invention is not limited to this. That is, when a user sings alone, entertainment can be improved by cutting out the appearance as partial video data only when the singer sings well.

(3) Others In the above embodiment and the modifications of (1) and (2), the position of the microphone 300 is specified based on the marker light of the microphone 300, and the partial video data including the singer who owns the microphone 300 is cut out. The process of correcting the partial video data is performed in real time during the singing of the singer, but is not limited thereto. That is, a video in the karaoke room KR obtained by photographing with the all-around camera 400 may be recorded, and the above processing may be executed after singing. Thereby, for example, a plurality of singers can sing all at once, and it is possible to give game characteristics such as uploading only the figure of a person who has successfully taken a picture or only the figure of a person who sang well.

  Moreover, in the said embodiment, although area A1-H4 which comprises the partial video data containing the singer who possesses the microphone 300 is determined automatically, it is not restricted to this. That is, the video in the karaoke room KR obtained by shooting with the all-around camera 400 is recorded, and after the singing of the singer, the user can select the areas A1 to H4 with the cursor keys or the like. Good. In this case, it is possible to realize a way of enjoying that a singer selects a video singing at a favorite angle, even though the karaoke store is visited alone.

  The flowcharts shown in FIG. 6, FIG. 13, FIG. 16 and the like do not limit the present invention to the procedure shown in the above-mentioned flow, and the addition / deletion or order of the procedures within the scope not departing from the gist and technical idea of the invention. May be changed.

  In addition to those already described above, the methods according to the above-described embodiments and modifications may be used in appropriate combination.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

DESCRIPTION OF SYMBOLS 10 Karaoke apparatus 100 Apparatus main body 101 Control part 106 Sound source (music reproduction means)
107 Voice control unit (music playback means)
108 Speaker (music playback means)
109 Display section (display means)
200 remote control 300 microphone (microphone)
300A microphone (microphone)
300B microphone (microphone)
304 LED (sign signal generating means)
400 All around camera (video camera)

Claims (4)

A karaoke apparatus that provides a reproduction service of karaoke performance music using music data and video data,
Music playback means for playing back the music data;
Display means capable of displaying the video data as the music data is played back by the music playback means;
A microphone possessed by a singer and for inputting a voice signal of karaoke singing by the singer;
A sign signal generating means for generating a sign signal provided in the microphone;
One moving image capturing camera that captures a predetermined range of visual field including the microphone and the singer, and generates the predetermined range of video data including the beacon signal generated from the beacon signal generation unit;
The position of the microphone is specified based on the indicator signal included in the video data of the predetermined range generated by the one video camera, and the partial video data including the specified position from the video data of the predetermined range Video extraction means for cutting out
A karaoke apparatus comprising: display control means for performing predetermined correction processing on the partial video data cut out by the video extraction means and displaying the video after the correction processing on the display means. The karaoke apparatus according to claim 1,
Voice determination means for determining whether an input level of a voice signal input from the microphone is a predetermined value or more;
The video extraction means includes
The input level is determined to be greater than or equal to a predetermined value, and the marker signal from the marker signal generating means identifies the position of the microphone included in the video data of the predetermined range. A karaoke apparatus characterized by cutting out partial video data including a specified position. In the karaoke apparatus according to claim 1 or 2,
A plurality of indicator signal generating means provided in each of the plurality of microphones generates the indicator signals of different colors;
The video extraction means includes
Distinguishing and specifying the position of each of the plurality of microphones based on the color of the indicator signal included in the predetermined range of video data, and cutting out the partial video data at each specified position from the predetermined range of video data. A karaoke device that features it. The karaoke apparatus according to any one of claims 1 to 3,
When the music data is played back by the music playback means, scoring means for scoring the voice signal of the karaoke song input from the microphone for each microphone;
The video extraction means includes
The position of the specific microphone whose scoring result by the scoring means is greater than or equal to a predetermined value is specified using the marker signal, and the partial video data at the specified position is cut out from the video data in the predetermined range. Karaoke equipment.

Images