JP7599181B2 - Hall display system and event implementation method using the same - Google Patents

Description

The present invention relates to a hall display system and a method for implementing an event using the same. More specifically, the present invention relates to a hall display system that includes a main screen that displays an animated character operating in the hall as a main character image, a sub-screen that is arranged in the hall separately from the main screen and is capable of displaying an image different from that of the main screen, and a computer that generates the images of the main screen and the sub-screen, and a method for implementing an event using the same.

As an example of a hall display system that displays 3D images and a method for implementing an event using this system, one such system is disclosed in the following Patent Document 1. This theater system aims to achieve a sense of realism by combining a holographic front screen and rear screen on the main screen. Meanwhile, in recent years, in pursuit of even greater realism, attempts have been made to improve the sense of realism by setting up a sub-screen next to the main screen and displaying characters at different angles, for example.

Special table 2017-509027 publication

In light of this current situation, the present invention aims to provide a hall display system that further enhances the sense of realism, and a method for implementing an event using the same.

In order to achieve the above object, the hall display system according to the present invention is characterized in that it comprises a main screen that displays an animated character operating in the hall as a main character image, a sub-screen that is placed in the hall separately from the main screen and is capable of displaying an image different from that of the main screen, and a computer that generates images for the main screen and the sub-screen, further comprising a first camera that captures the audience seats of the spectators watching both screens, the character has a second camera that is displayed as an image, the range of the character that falls within the virtual field of view of the second camera is generated as an image in the field of view of the second camera, and the generated image is displayed on the sub-screen, and further comprises a space in which the actor operates, a marker that is attached to the body of the actor, and a motion capture device that captures the movements of the actor with the marker, in a place not visible from the audience seats, and the computer has a rendering function that generates a character image in real time that is linked to the captured movements of the actor, the second camera generates an image based on the marker of the actor that exists in the vicinity of where the camera is displayed, and the virtual field of view is determined based on the marker.

According to the above characteristic configuration, the audience appears in the image of the auditorium captured by the first camera. Meanwhile, the character carries a second camera that is displayed as an image. Therefore, when the character makes a movement to capture the audience (audience) with the second camera that he or she carries, the range of the character that falls within the virtual field of view of the second camera is generated as an image in the field of view of the second camera. The generated image is then displayed on the sub-screen, improving the sense of realism and increasing the sense of unity between the character and the audience.

In the above configuration, the system further comprises, in a location not visible from the audience seats, a space in which the actor moves, a marker attached to the actor's body, and a motion capture device that captures the actor's movements using the marker, the computer has a rendering function that generates in real time a character image that is linked to the captured actor's movements, the second camera generates an image based on the actor's marker located in the vicinity of where the camera is displayed, and the virtual field of view is determined based on the marker. This reduces the sense of incongruity caused by differences between the main character image and the selected image captured by the second camera of the character, increasing the reality of the selected image and further enhancing the sense of presence.

In any of the above configurations, it is preferable that the sub-screen displays an image of the audience in the virtual field of view of the second camera, and the range of the audience that falls within the virtual field of view of the second camera is selected as a selected image from the image captured by the first camera, and the sub-screen displays both the image of the audience in the virtual field of view of the second camera and an image of the character that is included in the virtual field (so-called self-portrait image). In particular, the self-portrait image shows the audience together with the character, enhancing the sense of unity and liveliness.

In addition to any of the above configurations, it is preferable that the image of the audience displayed on the sub-screen is an image from a past time (T0-Δt) delayed by an infinitesimal time Δt from the real time T0 that is the reference for displaying the main character image, and the image on the sub-screen is an image from a past time (T0-Δt) delayed by an infinitesimal time Δt from the real time T0. This allows the audience to simultaneously see the main character image from real time T0 and a selected image from a past time (T0-Δt). Seeing the real time and the past time simultaneously, and seeing the delayed image on the sub-screen, shifts people's attention from the center to the sub-screen, which has the effect of making the audience feel a greater sense of spatial expansion and realism.

The main screen preferably comprises a hologram playback screen that is positioned at an angle above the stage and displays hologram images, and a rear screen that is positioned behind the stage, and displays a composite image of both screens to the audience.

The sub-screens are preferably positioned at least to the left and right and/or above the main screen. This allows for left and right and/or right-left spread.

In addition, in order to achieve the above-mentioned object, a feature of the method of implementing an event according to the present invention is that the method of implementing an event using a hall display system comprising a main screen which displays an animated character operating in a hall as a main character image, a sub-screen which is arranged in the hall separately from the main screen and which is capable of displaying an image different from that of the main screen, and a computer which generates images for the main screen and the sub-screen, further comprises a first camera which photographs the audience seats of spectators watching both screens, the character has a second camera which is displayed as an image, the range of the character which falls within the virtual field of view of the second camera is generated as an image in the direction of the field of view of the second camera, and the generated image is displayed on the sub-screen, and further comprises, in a place not visible from the audience seats, a space in which an actor operates, a marker which is attached to the body of the actor, and a motion capture device which captures the movements of the actor with the marker, the computer has a rendering function which generates in real time a character image which is linked to the captured movements of the actor, the image of the second camera is generated based on the marker of the actor which is located in the vicinity where the camera is displayed, and the virtual field of view is determined based on the marker.

The above-mentioned features of the hall display system and event implementation method using the same according to the present invention make it possible to provide a hall display system and event implementation method using the same that provide an even more realistic feeling.

Other objects, configurations and advantages of the present invention will become apparent from the following detailed description of the preferred embodiments of the invention.

FIG. 2 is a perspective view showing the configuration of a hall according to the present invention. FIG. 2 is a side view showing a schematic configuration around the stage. FIG. 1 is a diagram showing a system configuration of the present invention. FIG. 13 is a diagram showing an image on a main screen. FIG. 13 is a diagram showing a captured image projected on a sub-screen. FIG. 13 is a diagram showing a self-portrait image displayed on a sub-screen. FIG. 13 is a diagram showing the relationship between the posture of a character on the main and sub screens and time. FIG. 13 shows the state of an actor when the character takes a photo of the audience. FIG. 13 is a diagram showing a character photographing the audience. A diagram showing the relationship between video of the audience and photographed images of characters. FIG. 11 is a flow diagram for when a character takes a photo of the audience. FIG. 6B is a diagram equivalent to FIG. 6A when a character takes a selfie. FIG. 6B is a diagram equivalent to FIG. 6B when a character takes a selfie. FIG. 13 is a diagram showing a reflected image of a character. FIG. 11 is a flow diagram for when a character takes a selfie. FIG. 11 is a perspective view showing a second embodiment of the configuration in the hall. FIG. 11 is a perspective view showing a third embodiment of the configuration in the hall. FIG. 11 is a perspective view showing a fourth embodiment of the configuration in the hall. 13A is a side view showing a schematic configuration around a stage according to a fifth embodiment, and FIG. 13B is a plan view of FIG. FIG. 13 is a plan view showing a schematic configuration around a stage according to a sixth embodiment. A plan view showing the schematic configuration of the stage and audience seats according to a seventh embodiment.

Next, the present invention will be described in more detail with reference to the accompanying drawings as appropriate.
As shown in Figures 1 to 3, the hall display system 1 of the present invention roughly comprises a main screen 2, a pair of left and right sub-screens 3, lighting fixtures 4, speakers 6, a first camera 11 for capturing images of the audience seats AU of spectators watching both screens 2 and 3, and a computer 50 located in or near the hall H.

The hall display system 1 further includes a third camera 13 and a microphone 14 in a room separate from the hall H. The third camera 13 is installed in an isolated space S that is not visible from the audience seats AU, and captures the movements of an actor AC who is wearing multiple markers m on his body, and sends the footage to a motion capture module MC51. The microphone 14 records the voice of the voice actor or vocalist and sends the footage to a lip sync module LS54.

The main screen (main screen device) 2 installed in Hall H is equipped with a hologram reproduction screen 2a installed diagonally on the stage ST, a rear screen 2b installed behind the stage ST, a hologram projector 2c that projects images onto the hologram reproduction screen 2a, and a rear projector 2d that projects images onto the rear screen 2b. The image on the rear screen 2b passes through the hologram reproduction screen 2a and is composited with the image on the hologram reproduction screen 2a and is visible from the audience seats AU.

On the other hand, each sub-screen (sub-screen device) 3 is equipped with a first horizontal screen 3a1 and a second horizontal screen 3a2 (horizontal screen 3a) on the left and right, and a first sub-screen projector 3b1 and a second sub-screen projector (the latter is not shown, both collectively referred to as the sub-screen projector 3b) that project images onto these horizontal screens 3a, respectively.

The first camera 11 is, for example, positioned (fixed) facing the audience AU directly from the front at a position closer to the audience AU than the hologram reproduction screen 2a and the rear screen 2b, and captures a range (field of view) that includes roughly the entire audience AU. Since the entire audience AU is captured from directly in front of the audience AU, differences and distortions from the audience AU image BSI in the selected image MSI described below are unlikely to occur, and the sense of incongruity in the selected image MSI is also eliminated. In addition, since the first camera 11 covers most of the audience AU, the portion (range) of the audience AU that is not captured in the selected image is reduced, giving more spectators an opportunity to be captured in the selected image, and increasing the sense of realism and unity. For example, a wide-angle camera is used as the first camera 11. In addition, in the present invention, the character CH has a virtual camera CA as a second camera and performs the operation of capturing the audience AU.

The computer 50 includes a motion capture module MC51, a real-time rendering module RR52, a video synthesis module IM53, a lip sync module LS54, a backstage image generation module BI55, a virtual light image generation module VL56, a real light controller RC58, a music generation module SG59, and a time management controller TM60 that serves as the basis for synchronizing the time of each module.

The motion capture module MC51 analyzes the movements of the actor AC captured by the third camera 13, and generates a 3D image of the moving character in the real-time rendering module RR52. Multiple third cameras 13 are provided, and capture images of the actor AC from different angles to perform motion capture. Normally, the shooting direction of the front third camera 13 becomes the basic viewing direction D1 when generating the main character.

The virtual light image generation module VL56 creates an image of the lighting in the virtual space, and in conjunction with the real light controller RC58, sends light information to the backstage image generation module BI55 and the real-time rendering module RR52 to generate a shaded image consistent with the lighting conditions. The real lighting control unit RL61 is controlled by the real light controller RC58 and operates the real lighting fixtures 4 to provide real lighting in the hall H.

The image BSI of the audience AU captured by the first camera 11 is trimmed (selected) by the image synthesis module IM53 to the range of the virtual field of view VS of the virtual camera CA, and projected by the sub-screen projector 3b on the horizontal screen 3a as the captured image MSI1 of the character CH. In the case of a selfie, which will be described later, the image of the audience AU trimmed in the same manner as above (sub-background image BSI2, which will be described later) is further composited with the sub-character image SI of the real-time rendering module RR52, and projected by the sub-screen projector 3b on the horizontal screen 3a as the selfie image MSI2 of the character CH. Meanwhile, the image of the real-time rendering module RR52 is projected as the main character image MI on the hologram playback screen 2a and rear screen 2b by the hologram projector 2c and rear projector 2d, respectively.

The lip sync module LS54 delays the voice of the voice actor, etc., recorded by the microphone 14 to match the main character image. In addition, the music generation module SG59 generates background music and sound effects, which are then sent to Hall H by the speaker 6 via the sound system 62 along with the voice from the microphone 14.

As shown in Figures 1 and 5, a character CH (CH-A with hair, CH-B without hair) is displayed on the main screen 2. In contrast, a pair of left and right sub-screens 3 display images MSI1 and MSI2 of the character CH (CH-A, CH-B) captured by a virtual camera CA as a selected image MSI, as shown in Figures 1, 4B and C. Note that in the example shown in the figure, different images are displayed on the pair of left and right sub-screens 3, but of course the same selected image MSI may also be displayed.

To elaborate on the image on the main screen 2, as shown in FIG. 4A, the main character image MI is made up of the main character main body image MI1 and the main character's shadow MI2. A main background image BMI is further added to this to create the main composite image MMI.

In the example shown in FIG. 4B, on the first horizontal screen 3a1, an image of the audience AU taken by the character CH is displayed as a shot image MSI1. In the example shown in FIG. 4C, on the second horizontal screen 3a2, a selfie image MSI2 taken by the character CH with the audience AU in the background is displayed. In this selfie image MSI2, the sub-character image SI is made up of the sub-character main body image SI1 and the sub-character's shadow SI2. Then, by adding a sub-background image (part of the image of the audience AU) BSI2, the selfie image MSI2 is created.

The main composite image MMI on the main screen 2 uses a character rendered in the basic direction view D1. This basic direction view D1 may or may not match the direction view of the third camera 13. On the other hand, the direction view D2 different from the basic direction view D1 may be, for example, the direction view D3 of the first camera 11. Therefore, the character CH taking the selfie faces backwards towards the audience AU on the main screen 2, but faces forward on the sub-screen 3.

Now, the generation of the photographed image MSI1 of the character CH will be described in detail with reference to FIGS.
When the character CH (CH-A) points the second camera, which is a virtual camera CA, in front of his/her body toward the audience AU, only the audience AU is captured in the image. The image of the first camera 11 is used for the image BSI of the audience AU (steps S11 and S12).

For example, when a character CH (CH-A) takes a picture by holding a smartphone SP in his/her hand as a second camera, which is a virtual camera CA, as shown in Figs. 6A and 6B, the position (x, y, z) of the virtual camera CA is calculated from the set dimensions of the smartphone SP and the character CH based on the position (x0, y0, z0) of the marker m0 on the arm of the actor AC, which is closest to the virtual camera CA. The actor AC does not need to actually hold the smartphone. Then, by assigning a virtual field of view VS to the position of the camera CA, the trimming range TR (TR1) in the image BSI of the first camera 11 is determined as shown in Fig. 6C. The trimming range TR is the planar position information of the image BSI, and is therefore expressed by the x coordinate (the width (horizontal) direction of the stage ST) and the z coordinate (vertical direction) (step S13). Then, the image MSI1 (step S14) trimmed based on this planar position information is displayed on the sub-screen 3 (step S15).

Next, the generation of the selfie image MSI2 of the character CH (CH-B) will be described in detail with reference to FIGS.
When the character CH takes a selfie with his/her back facing the audience AU, the image shows the audience AU and part of the character CH's body. As described above, the image BSI from the first camera 11 is trimmed (steps S11 to S14) in the trimming range TR (TR2) in Fig. 6C for the sub background image BSI2 serving as the image of the audience AU.

Here, the position (x', y', z') of the camera CA is calculated based on the position (x0', y0', z0') of the reference marker m0' as described above (step S21). In addition, the position information of the reflected body is three-dimensional position information including the position information of the arm, face and body in the example of FIG. 8C, and is obtained by adding the y coordinate (front and rear direction of the stage ST) to the previous x and z coordinates (steps S22, S23). This position information is based on the position information of each marker m of the actor AC. Then, based on this three-dimensional position information, the part (range) of the character CH reflected in the virtual field of view VS of the virtual camera CA is determined (step S24), and the part is rendered to generate the sub-character image SI (step S25). Then, the self-portrait image MSI2 (step S26) in which the trimmed sub-background image BSI2 and the rendered sub-character image SI of the character CH are combined is displayed on the sub-screen 3 (step S27).

In FIG. 1, real illumination light L1 is emitted from the real lighting device 4 onto the audience seats AU, the stage ST, and the characters CH.

As shown in Figures 1, 3 to 5, the main screen 2 displays the character CH at real time T0. In contrast, the sub-screen 3 displays images MSI1 and 2 captured by the character CH at a time (T0-Δt) going back an infinitesimal time Δt. The images on each screen have shadows and backgrounds generated based on information about the real illumination light L1 from the real lighting fixture 4 that is set to each time.

The images BSI1,2 of the audience AU in the images MSI1,2 also use images from a past time (T0-Δt) going back an infinitesimal time Δt. In particular, in the case of the selfie image MSI2, the sub-character image SI also uses an image from a past time (T0-Δt), so there are no inconsistencies such as different lighting colors, and the audience accepts the image on the sub-screen 3 with a very natural sensation, resulting in a natural sense of realism. In the example of Figure 5, the character CH at real time (T0) is in the process of turning his body from the selfie pose at the past time (T0-Δt) towards the audience AU, and the character CH enclosed in a dashed circle in the figure is projected on the screen.

However, the images BSI1 and 2 of the audience seating AU may be delayed by a small time Δt due to the performance of the hardware. In the case of the selfie image MSI2, the sub-character image SI may use an image going back by the small time Δt depending on the performance of the hardware, and the small time Δt may be adjusted by adjusting the hardware delay with the software delay of the image synthesis module IM53. The small time Δt is preferably set to 0.1 to 0.5 seconds for optimal implementation. If it is 0.1 seconds or more, the difference in the images can be recognized, but if it exceeds 0.5 seconds it will appear unnatural.

A main character main body image MI1, a main character shadow MI2, and a main background image BMI are synthesized and displayed on the main screen 2. These elements can be displayed on the hologram reproduction screen 2a and the rear screen 2b in the following combinations.
Case 1) Hologram playback screen 2a: main character body image MI1, rear screen 2b: main character shadow MI2 and main background image BMI
Case 2) Hologram playback screen 2a: main character body image MI1 and main character shadow MI2, rear screen 2b: main background image BMI
Case 3) Hologram playback screen 2a: main character body image MI1, main character shadow MI2, and part of main background image BMI (BMI1), rear screen 2b: remainder of main background image BMI (BMI2)
The screen layout of these images will be adapted as appropriate according to need.

In the above, a space in which the actor AC can operate and a motion capture device are provided. However, as shown in FIG. 3, the event may be carried out by recording all of the controls for the video, audio, lighting, etc., for the event in the recording module RS57 and playing them back.

Furthermore, selected images such as the selfie image MSI2 recorded in the recording module RS57 may be sent (distributed) to, for example, mobile devices of spectators in the audience seating AU, or posted (transmitted) in real time on a network such as a social networking service (SNS). This allows a sense of unity to be felt with fans other than the spectators in the audience seating AU.

Next, another embodiment of the present invention will be described. The same members as those in the first embodiment are designated by the same reference numerals.
In the above embodiment, the virtual camera CA is a smartphone SP, but the present invention is not limited to this and can be applied to various types of photographing devices. Also, although an example in which the smartphone SP is held in the hand is shown, it may be another part of the character CH, or a photographing method using another photographing tool such as a selfie stick may be used.

In the above embodiment, the first camera 11 is placed between the audience seats AU and the stage ST. However, the position of the first camera 11 is not limited to this, so long as it can capture an image of almost the entire audience seats AU. For example, the first camera 11 may be fixed above or behind the main screen 2 (hologram reproduction screen 2a) so as to capture an image of the entire audience seats AU from directly in front of the audience seats AU. In this position, the position of the first camera 11 and the position of the character CH projected on the main screen 2 will be almost the same relative to the audience seats AU, increasing the realism of the selfie image MSI2. However, care must be taken to ensure that the character CH projected on the main screen 2 is not reflected in the first camera 11.

In the above embodiment, a pair of left and right horizontal screens 3a (3a1, 3a2) are provided as the sub-screens 3, but as shown in FIG. 10, an upper screen 3c may be additionally provided above the main screen 2. In this example, the image in the trimming range TR3 in FIG. 6c is used. Also, as shown in FIG. 11, instead of providing a pair of left and right horizontal screens 3a (3a1, 3a2), a pair of left and right upper screens 3c (3c1, 3c2) may be provided. Each of these upper screens 3c is provided with a corresponding projector, but this is omitted from the illustration and will be the same hereinafter.

In the above embodiment, the main composite image MMI projected on the main screen 2 is the real time T0, and the selected images MSI projected on the left and right sub-screens are the past time (T0-Δt), with two stages of time difference between the screens. However, the time between the three screens may be changed to three stages. For example, the real time may be used for the main screen 2, the future time for one sub-screen 3, and the past time for the other sub-screen 3, or multiple times such as the present, past, and further past may be used. The number of screens and the time switching may be three or more.

In another embodiment of FIG. 12, in addition to the pair of left and right horizontal screens 3a (3a1, 3a2), a pair of left and right upper screens 3c (3c1, 3c2) are provided. The pair of left and right upper screens 3c (3c1, 3c2) display past images (MSI1', MSI2') from time T0-2Δt, and the pair of left and right horizontal screens 3a (3a1, 3a2) display past images (MSI1, MSI2) from time T0-Δt. In addition, the times of the above horizontal screens 3a and upper screens 3c can be reversed, all sub-screens can be set to the same time, or the time of the sub-screens can be changed and selected as appropriate.

In another embodiment of FIG. 13, a further pair of the above-mentioned main screen 2 and sub-screen 3 are provided back-to-back on the stage ST. For example, the hologram reproduction screen 2a is provided with a first set of members whose final letters are a and a second set whose final letters are b, as indicated by the symbols 2aa and 2ab. In addition, the pair of left and right first horizontal screens 3aa (3a1a, 3a2a) and the pair of left and right second horizontal screens 3ab (3a1b, 3a2b) are each provided with a screen drive device 70 that moves the horizontal screen up, down, left and right, and moves the sub-screen 3a up, down and left and right in accordance with the sound and image.

In another embodiment of FIG. 14, four sets of the above-mentioned main screen 2 and sub-screen 3 (only 3c) are provided on the stage ST, back-to-back and shifted by 90 degrees each. In the first to fourth sets, the last alphabet of the code is a to d to identify each set. In this arrangement, the audience can see the screens 2 and 3 from a 360-degree range around them, and by providing these stages ST in the center of an arena, for example, it is possible to seat more audience members closer to the main screen 2. In this case, in addition to projecting images at the same time on the first to fourth sets, the first to fourth sets may also project images at different times.

In another embodiment of FIG. 15, in addition to the pair of left and right first horizontal screens 3a (3a1, 3a2) provided beside the stage ST, a pair of left and right second horizontal screens 3ax (3a1x, 3a2x) and a third horizontal screen 3ay (3a1y, 3a2y) may be provided in sequence closer to the audience seats 80. The time of the images on each of these horizontal screens can be changed as appropriate according to the combinations described above.

The present invention can be used as a hall display system that can be used for concerts and events featuring virtual characters, and as a method for implementing events using the system. The hall can be outdoors, and any hall can be used as long as a screen can be installed on the stage.

AU: Audience, CH (CH-A, CH-B, CH-C): Character, H: Hall, ST: Stage, AC: Actor, S: Isolated space, D1: View in basic direction, D2: View in direction different from basic direction, D3: View in first camera direction, MI (MIa, MIb): Main character image, MI1: Main character main image, MI2: Main character shadow, BMI: Main background image, MMI: Main composite image, SI: Sub character image, SI1: Sub character main character image, SI2: shadow of sub-character, BSI: image of audience, MSI (MSI1, MSI2, MSI1', MSI2'): selected image, MSI1: shot image, MSI2: self-shot image, L1: actual lighting, T0: actual time, Δt: minute time, T0-Δt: past time, m0: reference marker, m: marker, VS: virtual field of view, CA: virtual camera (second camera), TR: trimming range, 1: hall display system, 2: main screen, 2a: hologram playback screen 2b: Rear screen, 2c: Hologram projector, 2d: Rear projector, 3: Sub-screen, 3a: Horizontal screen, 3a1: First horizontal screen, 3a2: Second horizontal screen, 3b: Sub-screen projector, 3b1: First sub-screen projector, 4: Real lighting equipment, 6: Speaker, 11: First camera, 13: Third camera, 14: Microphone, 50: Computer, MC51: Motion capture module, RR52: Real-time rendering module, IM53: Image synthesis module, LS54: Lip sync module, BI55: Backstage image generation module, VL56: Virtual light image generation module, VS (VS1, VS2, VS3): Virtual field of view, RS57: Recording module, RC58: Real light controller, SG59: Music generation module, TM60: Time management controller, RL61: Real lighting control unit, 62: Sound system

Claims (7)

A main screen that displays the animated characters moving in the hall as main character images;
a sub-screen that is disposed in the hall separately from the main screen and is capable of displaying an image different from that of the main screen;
A hall display system including a computer that generates images of the main screen and the sub-screen,
A first camera is further provided for photographing the audience seating area of the audience watching both screens,
The character has a second camera that is displayed as an image,
A range of the character that falls within a virtual field of view of the second camera is generated as an image in a field of view direction of the second camera;
Displaying the generated image on the sub-screen;
The system further includes a space for an actor to move in a place not visible from the audience seats, a marker to be attached to the body of the actor, and a motion capture device to capture the movement of the actor by the marker,
The computer has a rendering function that generates character images in real time that are linked to the captured actor's movements,
A hole display system in which the second camera is generated as an image based on a marker of the actor located in the vicinity of where the camera is displayed, and the virtual field of view is determined based on the marker. The hall display system of claim 1, wherein the sub-screen displays an image of the audience seats in the virtual field of view of the second camera. A display system for a hall as described in claim 1, in which the range of the audience seats that falls within the virtual field of view of the second camera is selected as a selected image from the image captured by the first camera, and the image of the audience seats in the virtual field of view of the second camera and the image of the character included in the virtual field of view are both displayed on the sub-screen. A hall display system according to any one of claims 1 to 3, in which the image of the audience displayed on the sub-screen is an image from a past time (T0-Δt) delayed by a minute time Δt from the real time T0 that is the reference for displaying the main character image, and the image on the sub-screen is an image from a past time (T0-Δt) delayed by a minute time Δt from the real time T0. The hall display system according to any one of claims 1 to 4, wherein the main screen comprises a hologram playback screen that is arranged at an angle above the stage and displays hologram images, and a rear screen that is arranged behind the stage, and displays a composite image of both screens to the audience. A hall display system according to any one of claims 1 to 5, wherein the sub-screen is disposed at least to the left and right and/or above the main screen. A main screen that displays the animated characters moving in the hall as main character images;
a sub-screen that is disposed in the hall separately from the main screen and is capable of displaying an image different from that of the main screen;
A method for implementing an event using a hall display system including the main screen and a computer for generating images on the sub-screen, comprising:
A first camera is further provided for photographing the audience seating area of the audience watching both screens,
The character has a second camera that is displayed as an image,
A range of the character that falls within a virtual field of view of the second camera is generated as an image in a field of view direction of the second camera;
The generated image is displayed on the sub-screen, and the system further includes a space for an actor to move, a marker attached to the body of the actor, and a motion capture device that captures the movement of the actor using the marker, in a location not visible from the audience seats;
The computer has a rendering function that generates character images in real time that are linked to the captured actor's movements,
A method for implementing an event in which the second camera is generated as an image based on a marker of the actor located in the vicinity in which the camera is displayed, and the virtual field of view is determined based on the marker.

Images