JP6945312B2 - Operation control system, character screening system and program - Google Patents

Description

The present invention relates to an operation control system, a character screening system, a program, and the like.

Conventionally, at an event venue that accommodates multiple spectators, a system for showing non-existent 2D or 3D characters such as idols to the spectators and operating the characters as if they were there ( Hereinafter, it is also referred to as an "event character operation system").

On the other hand, in a moving image editing device for editing a CG animation or a moving image, a device that eliminates complicated image editing by an operator is known (for example, Patent Document 1). Then, by incorporating such a moving image editing device into the character operation system for an event, it is considered to edit and provide a moving image including a character to be screened in real time according to the situation of the audience or the venue. There is.

Japanese Unexamined Patent Publication No. 2001-195600

However, even if the video editing device described in Patent Document 1 is simply incorporated into the character operation system for an event, the operation of the operator can be simplified, but the character can be operated according to the atmosphere of the venue and the state of the audience. It is difficult to control the operation.

The present invention has been made to solve the above problems, and an object of the present invention is to move a character and image it according to the atmosphere of an event venue and the state of an audience while simplifying the operation by an operator. It is an object of the present invention to provide a character operation system capable of improving the presence of a character and the live feeling in the performance of the character, enlivening the event, and executing the event with high satisfaction in the audience.

(1) In order to solve the above problems, the present invention
It is an operation control system that controls the operation of the imaged target character to be viewed by the audience.
An image generation means that sets a virtual camera in the object space in which the target character is arranged and generates an image of the target character in the object space with the virtual camera as a viewpoint.
A spectator display control means for displaying an image of the target character on the spectator display device visually recognized by the spectator, and
From a plurality of operation commands for controlling the operation of the target character in the object space, an operation command selected by the operator according to the spectator information indicating information about the spectator is used as an operation command candidate by the operator. Presenting means to present and
A receiving means for receiving an operation command instructed by the operator from the presented operation command candidates, and
An operation control means for controlling the operation of the displayed target character based on the received operation command, and
Has a configuration.

With this configuration, the present invention provides information indicating the atmosphere of the spectator and the current state of the spectator, or an operation command selected by the operator according to the spectator information of each spectator such as the attributes of the spectator and the billing history, or An operation command extracted according to the spectator information and selected by the operator can be presented to the operator as an operation command candidate.

For example, in the present invention, when it is determined from the audience information that the spectator who is observing the target character such as the event venue is not excited, the MC (line or microphone) is used as an operation for the target character to excite the spectator. The situation of the spectator who is observing the target character (that is, the current situation of the spectator), such as making it possible to select a command to execute a performance) or an operation command to execute a violent dance by the target character. The operator can be made to select the operation of the target character according to the state).

Further, for example, the present invention controls the movement of the target character for women when it is determined from the audience information that there are many women in the audience viewing the target character, or predetermined goods in past actions. If it is determined that the target character has been purchased, an operation command that controls the behavior of the target character according to the behavior history and attributes of the spectator who is observing by causing the spectator to perform a special effect of the target character. Can be selected by the operator.

Therefore, according to the present invention, the target character can be operated according to the atmosphere of the spectator and the state of the spectator while simplifying the operation by the operator. While improving the live feeling in the performance, it is possible to excite the audience and realize a fresh performance.

As a result, the present invention can control the movement (that is, performance) of the target character, which is highly satisfying to the audience.

The audience display device is, for example, a device provided on the stage and displaying a target character to be viewed for performing a performance such as a song, a dance, a play, or a lecture. For example, a liquid crystal display device or a projector. Such as a projection device, or a three-dimensional imaging device such as a hologram.

Further, the "operation command selected according to the spectator information" is, for example, an operation command that can be selected by the operator's intention while referring to the spectator information, and, for example, a camera. It is an operation command selected (extracted) according to the spectator information detected by the detection means such as, and indicates the operation command selected by the operator.

For example, in the audience information,
(1) Audience attribute information such as gender, age, residential area or viewing position (seat) (2) Audience preference information such as favorite acting characters,
(3) Information on terminal devices such as smartphones and ID tags owned by the spectator (for example, information indicating the position (seat) at the time of viewing such as information identifying the spectator's individual and electronic ticket information)
(4) Information that is stored in association with each spectator such as an ID or in association with a terminal device owned by the spectator, and the target character being viewed or the target character is performing. Information (game play information) regarding the play status and history (including billing history) of a predetermined game such as a game related to an event,
(5) Information stored using a terminal device owned by the spectator, which is related to the target character, action history information indicating participation in an event in which the target character is performing, and the event or performance. Billing information and billing history information from the designated online shopping site and actual sales stores (including inside the event venue), as well as
(6) Biological information (for example, face image, voice information (including voice print information of each spectator and frequency characteristic information indicating frequency characteristics), posture information, calorific value information, and a predetermined device (smartphone, etc.) from each spectator. Information similar to biological information such as light information using psyllium)) is included.

Further, the operation command may include a command for operating and controlling the operation of the imaged target character in the object space, and a command for executing a plurality of commands integrally.

For example, the plurality of operation commands may be a plurality of types of operation commands (jump, run, one-turn operation command, etc.) in which the operation of the target character is different, or the same type having different effects when executing the operation command. It may be an operation command of (such as an operation command that issues a different line).

In addition to the above, "presenting operation command candidates" may be a case of operably displaying on a tablet having a display device or a touch panel function, or an operation command is assigned to a physical switch or keyboard. It may be a case of indicating that it has been done.

Further, the "motion control of the target character" includes not only controlling the motion according to the operation command but also operating according to the operation command at a predetermined timing or the like while operating according to a predetermined algorithm.

In particular, when controlling the movement of the target character using a predetermined algorithm, it is shown that the movement is controlled by an algorithm that regulates the movement (including dialogue) at the time of performance such as singing or dancing. When an operation command is received while the operation is controlled, the operation control of the target character according to the operation command is included in place of the processing based on the algorithm or together with the processing of the algorithm.

(2) Further, the present invention
A monitor display control means for displaying the target character on the monitor display device visually recognized by the operator is further provided.
The presentation means
The monitor display device has a configuration in which the operation command candidate is presented to the operator together with the target character.

With this configuration, the present invention can display operation command candidates on a character that is a target of operation that can be visually recognized by the operator, so that the operation of the operator can be simplified.

In particular, in the present invention, when the monitor display device is composed of a touch panel, operation command candidates can be presented together with the target character whose operation is to be controlled, so that the operator's intuitive operation can be promoted and the operator's operation can be promoted. Operation input can be simplified.

The monitor display device indicates a monitor for an operator provided separately from the display device for spectators, and is composed of, for example, a tablet terminal device, a smartphone, a liquid crystal device, and the like. As described above, the monitor display device is preferably configured by a touch panel.

(3) Further, the present invention
The presentation means
It has a configuration in which the spectator information is acquired and the acquired spectator information is presented to the operator together with the operation command candidate.

With this configuration, the present invention allows the operator to recognize the spectator information when selecting the operation command candidate, so that the operator can issue an operation command in which an operation matching the spectator is defined for the target character. It can be made easier to select.

Therefore, the present invention can present spectator information such as the behavior history and attributes of the spectator to excite the spectator and provide fresh behavior, so that the movement of the target character with high satisfaction in the spectator can be provided. It is possible to make it easier to select an operation command for which (that is, performance) is specified.

(4) Further, the present invention
For each of the spectators, a management means for managing a storage means for storing the personal information of each spectator as the spectator information is further provided.
The presentation means
It has a configuration in which the stored spectator information is acquired and the acquired spectator information is presented to the operator together with the operation command candidate.

With this configuration, the present invention allows the operator to recognize the personal information of the spectator when selecting the operation command candidate, so that the target character can operate the operation command in which the operation matching the spectator is defined. It can be made easier for some people to choose.

For example, if it can be recognized that there are many women in the audience viewing the target character based on the presented personal information, the operator is made to select an operation command for controlling the operation of the target character for women. It will be easier, and if it can be recognized that there is a spectator (there are many spectators) such as purchasing certain goods in the past actions, the operator will be given a special effect of the target character. It becomes easier to select the operation command to be made.

That is, the present invention can present the spectator's personal information such as the spectator's behavior history and attributes to excite the spectator and provide a fresh behavior, so that the target character is highly satisfying to the spectator. It is possible to make it easier to select an operation command whose operation (that is, performance) is specified.

The personal information includes, for example, behavior history information indicating the past behavior history of each spectator, gender, age, residential area, favorite target character of each spectator, position (seat) at the time of viewing, or a smartphone. Attribute information such as information about terminal devices such as ID Doug (RFID) and ID Doug (RFID) is included.

In particular, the information related to the terminal device includes, for example, information that identifies an individual spectator, information that indicates a position (seat) at the time of viewing such as ticket information, and purchase history information such as game history information and products.

In addition, the action history information includes, for example, the number of times and the date and time of participating in a related event such as an event in which the target character is performing, a game related to the target character and the event in which the target character is performing, and the like. Information on the play status or history of the specified game, purchase information of products (goods) and services from the specified online shopping site and actual sales stores (including inside the event venue) related to the event and target characters, etc. The billing information (including billing history information) generated by this is included.

(5) Further, the present invention
Further provided with a registration means for registering and accumulating the information of the operation command received when the operation control of the target character is executed in the storage means as usage history information.
The presentation means
It has a configuration in which an operation command selected by the operator according to the spectator information and the registered usage history information is presented to the operator as the operation command candidate.

With this configuration, the present invention responds to usage history information such as frequency of use, type of operation command used immediately before, and information indicating whether or not it is used within a predetermined period (during a series of performance periods, etc.). Therefore, the operation command selected by the operator can be presented to the operator as an operation candidate command.

For example, the present invention defines an operation command that is frequently used, an operation command related to the immediately preceding operation command, and an operation command that defines the behavior of a target character that most of the spectators have not yet viewed when selecting an operation command. The operation commands that have not been used during the execution of a series of performances, or the operation commands that should be recognized by the operator as operation command candidates that can be selected preferentially over other operation commands are narrowed down and presented to the operator. be able to.

Further, for example, the present invention does not present non-selectable operation command candidates such as an operation command that has already been executed or an operation command that has executed a special effect already provided to a specific audience in a series of performances. can do.

Further, for example, according to the present invention, if the already registered operation command is analyzed, the operation command that regulates the behavior of the target character that is always excited, or the operation command that regulates the behavior of the target character that functions effectively in any scene, etc. , Operation commands that follow the tendency of past operation commands can be presented as operation command candidates.

Therefore, according to the present invention, the character can be operated according to the atmosphere of the spectator and the state of the spectator while further simplifying the operation by the operator by using the past usage history, and the target character of each spectator can be operated. It is possible to excite the audience and provide fresh behavior without compromising the interest in the viewing.

(6) Further, the present invention
The usage history information includes the operation command and spectator information used when the operation command is received.
The presentation means
An operation command selected by the operator according to a given analysis result in the operation command based on the usage history information and the spectator information is presented to the operator as the operation command candidate. It has a configuration.

With this configuration, the present invention is an operation command that defines the behavior of the target character that is always excited by analyzing it together with customer information using the already registered operation command, or the behavior of the target character that functions effectively in any scene. It is possible to present operation commands in line with the tendency of past operation commands, such as operation commands that specify the above, as operation command candidates.

(7) Further, the present invention
Further provided with a detection means for detecting the state of each spectator as the spectator information,
The presentation means
It has a configuration in which an operation command selected by the operator according to the detected spectator information is presented to the operator as an operation command candidate.

With this configuration, the present invention detects, for example, the spectator's characteristics such as the spectator's face image, sound, emitted light, or information of a terminal device owned by the spectator, or the spectator's reaction to the behavior of the spectator's target character. The characteristics of the spectators who are visiting the target character (for example, there are many spectators with a specific light or many women), or the state when the spectator is viewing the target character. It can be detected as spectator information.

Therefore, according to the present invention, it is possible to narrow down and present the operation command candidates according to whether or not the place where the target character is being viewed is excited by the audience information, or the state and attributes of the audience. While simplifying the operation in, it is possible to improve the sense of existence of the character to be imaged and the live feeling in the performance of the character, to excite the audience, and to perform a highly satisfying performance in the audience.

Further, according to the present invention, if the detected spectator information is accumulated, the movement of the target character and the state of the spectator are stored in a database and used for the subsequent movement of the target character (that is, the same performance at different times and different places). can do.

Therefore, according to the present invention, if a large amount of spectator information can be stored in a database, the spectators can be further excited and the spectators can perform more satisfying performances.

The detection means includes, for example, a camera or infrared sensor that recognizes the face of the spectator and detects light emitted by a device such as a psyllium or a smartphone owned by the spectator, a microphone that collects sounds such as cheering of the spectator, and the entire spectator. Thermography that detects the amount of heat of the spectator, a seating sensor that detects the position of each spectator (whether it is seated or standing up, etc.), a detection unit consisting of a marker unit and a camera, or information on a terminal device owned by the spectator ( For example, a wireless receiver that detects the viewing (seat) position of each spectator and information for identifying an individual) is included.

As spectator information indicating the state of each spectator, for example, ecological information such as spectator's face information and sound information (frequency information and volume information), or spectator's state such as calorific value information, posture information and light information. Information equivalent to recognizable biometric information is included.

(8) Further, the present invention
The reception means
Along with the above operation command, a spectator designation instruction for designating a specific spectator who enjoys the movement of the target character is received.
The motion control means
It has a configuration that controls the operation of the displayed target character toward the spectator specified by the received spectator designation instruction.

With this configuration, the present invention can control the movement of the target character such as a special effect for the specific spectator at the viewing position of the specific spectator. It is possible to improve the live feeling in the character's performance, excite the audience, and perform a highly satisfying performance in the audience.

The actions of the target character enjoyed by a specific spectator include throwing kisses, gestures, and issuing names and characteristic lines to the specific spectator.
Further, the designation of the viewing may be executed by using an operator such as a touch panel or a joystick using a display device that displays the position in the venue of each viewing, or the data of the face image of the audience is stored. It may be automatically specified by using a detection means such as a camera on the premise that it has been performed.

(9) Further, the present invention
The presentation means
It has a configuration in which an operation command determined according to the operation status of the target character is included in the operation command candidate and presented to the operator.

With this configuration, the present invention always performs an operation command that is expected to be used, such as the first action or the last action of the target character, or a predetermined timing such as singing a specific song based on the past usage history. Since the operation command to be used can be included in the operation command candidates and presented, suitable operation commands can be extracted as operation command candidates.

The "operation status of the target character" indicates the status at which each action of the target character, such as the start timing and the end timing of the performance, is at the start or during execution.

(10) Further, the present invention
The presentation means
The presentation according to the elapsed time from the start timing of image generation in the target character, the number of times the operation command received during the image generation is received, or the combination of the types of the operation command received during the image generation. It has a configuration that changes the type of operation command to be performed.

With this configuration, the present invention provides, for example, the movement of the target character at that time during the movement of the target character by an algorithm for controlling a series of movements of the target character performing a performance such as a song, a dance, a play, or a lecture. It is possible to present an operation command suitable for the operator to the operator.

For example, the present invention
(1) At the start timing of image generation (that is, screening of contents) in the target character, an operation command for controlling an action for enlivening the audience is presented.
(2) In the latter half of the screening of the content, an operation command for maintaining it and controlling the behavior for further excitement is presented to the enthusiastic audience.
(3) For quiet scenes such as between songs, present operation commands specific to that scene, such as "line".
be able to.

Therefore, the present invention can improve the presence of the character to be imaged and the live feeling in the performance of the character, excite the audience, and perform a highly satisfying performance in the audience.

(11) Further, the present invention
When the behavior of the target character placed in the object space is imaged while following a given algorithm,
For each operation command, the reception completion allowable time indicating the time when the reception of the operation command is permitted from the start timing of the image generation in the target character or a plurality of given sections is defined, and the image generation is executed. The section information indicating the section in which the operation command is allowed to be accepted and the execution time during which the operation control of the target character is executed based on each operation command are defined.
The presentation means
The type of the operation command to be presented is determined according to the elapsed time from the start timing of image generation in the target character, the section to which the timing for presenting the operation command belongs, and the execution time of each operation command. Alternatively, it has a configuration for presenting an operation command in which the content of the operation control of the target character is changed.

With this configuration, the elapsed time from the start timing of image generation, or, for example, when the motion control is a live performance in a music, the prelude, A melody, B melody or chorus, and in the case of a play, the first act or It is possible to identify which part of each part (section) such as Act 2 is the image and present the operation command, and depending on the situation, control the operation of the target character in the operation command such as shortening the time and simplifying the operation content. The content can also be changed.

Therefore, the present invention is suitable as the movement of the target character at that time during the movement of the target character by the algorithm for controlling a series of movements of the target character performing a performance such as a song, a dance, a play, or a lecture. Operation commands can be presented to the operator.

(12) Further, the present invention
The motion control means
When the operation command is received during image generation in the target character, the operation control of the displayed target character based on the received operation command is started from a predetermined timing. There is.

With this configuration, the present invention can control the operation of the target character in the operation command by waiting for the switching timing of the operation of the target character during image generation (that is, during the screening of the content) of the target character. On the other hand, seamless operation control can be performed without any discomfort.

(13) Further, the present invention
The presentation means
While presenting the operation command candidate, it has a configuration in which the operation start time at which the operation control of the target character is started or the time related to the operation start time is presented based on the operation command received at the timing.

With this configuration, the present invention can prevent the operator from forgetting to input and improve the operability of the operator.

The "time related to the operation start time" includes, for example, the remaining time from the current timing to the operation start time.

(14) Further, the present invention
The motion control means
When the operation of the target character arranged in the object space is imaged while following a given algorithm, and the operation command is accepted, the algorithm after accepting the operation command is described as described above. It has a configuration that changes according to the received operation command and controls the operation of the displayed target character according to the changed algorithm.

With this configuration, the present invention controls the operation of the target character in accordance with the reception of the operation command. Therefore, it is possible to provide an image of the target character whose operation is controlled by the operator, and every time the operation command is received. In addition, the behavior of the target character can also be changed.

Further, according to the present invention, the algorithm for controlling the operation of the target character can be changed even when the performance time of the target character is changed by the operation control by the operation command. You can develop a story of content such as performance without compromising.

Therefore, according to the present invention, the target character can be operated according to the atmosphere of the spectator and the state of the spectator while simplifying the operation by the operator. While improving the live feeling in the performance, it is possible to excite the audience and realize a fresh performance.

(15) Further, in order to solve the above problems, the present invention
It is a program that controls the operation of the imaged target character to be viewed by the audience.
An image generation means that sets a virtual camera in the object space in which the target character is arranged and generates an image of the target character in the object space with the virtual camera as a viewpoint.
A spectator display control means for displaying an image of the target character on a spectator display device visually recognized by the spectator.
From a plurality of operation commands for controlling the operation of the target character in the object space, an operation command selected by the operator according to the spectator information indicating information about the spectator is used as an operation command candidate by the operator. Presenting means to present and
A receiving means for receiving an operation command instructed by the operator from the presented operation command candidates, and a receiving means for receiving the operation command.
An operation control means that controls the operation of the displayed target character based on the received operation command.
It has a configuration that makes the computer function as.

With this configuration, the present invention provides information indicating the atmosphere of the spectator and the current state of the spectator, or an operation command selected by the operator according to the spectator information of each spectator such as the attributes of the spectator and the billing history, or An operation command extracted according to the spectator information and selected by the operator can be presented to the operator as an operation command candidate.

Therefore, according to the present invention, the target character can be operated according to the atmosphere of the spectator and the state of the spectator while simplifying the operation by the operator. While improving the live feeling in the performance, it is possible to excite the audience and realize a fresh performance.

As a result, the present invention can control the movement (that is, performance) of the target character, which is highly satisfying to the audience.

(16) Further, in order to solve the above problems, the present invention
An operation control system that controls the operation of the imaged target character to be viewed by the audience, and
A spectator display device visually recognized by the spectator,
With
The operation control system
An image generation means that sets a virtual camera in the object space in which the target character is arranged and generates an image of the target character in the object space with the virtual camera as a viewpoint.
A spectator display control means for displaying an image of the target character on the spectator display device, and
From a plurality of operation commands for controlling the operation of the target character in the object space, an operation command selected by the operator according to the spectator information indicating information about the spectator is used as an operation command candidate by the operator. Presenting means to present and
A receiving means for receiving an operation command instructed by the operator from the presented operation command candidates, and
An operation control means for controlling the operation of the displayed target character based on the received operation command, and
It has a configuration including.

With this configuration, the present invention provides information indicating the atmosphere of the spectator and the current state of the spectator, or an operation command selected by the operator according to the spectator information of each spectator such as the attributes of the spectator and the billing history, or An operation command extracted according to the spectator information and selected by the operator can be presented to the operator as an operation command candidate.

Therefore, according to the present invention, the target character can be operated according to the atmosphere of the spectator and the state of the spectator while simplifying the operation by the operator. While improving the live feeling in the performance, it is possible to excite the audience and realize a fresh performance.

As a result, the present invention can control the movement (that is, performance) of the target character, which is highly satisfying to the audience.

It is a block diagram which shows the outline structure in one Embodiment of the content screening system which concerns on this invention. It is a figure for demonstrating the relationship between the content screening system and a large-sized display device in the event venue of one Embodiment, and the relationship between a performance character and an audience. It is an example of the functional block diagram which shows the structure of the server apparatus in one Embodiment. It is an example of the functional block diagram which shows the structure of the system control device in one Embodiment. It is a figure (the 1) for demonstrating the operation command candidate presenting process of the content of one Embodiment. It is a figure (the 2) for demonstrating the operation command candidate presenting process of the content of one Embodiment. It is a figure (3) for demonstrating operation command candidate presentation processing of the content of one Embodiment. It is a figure (the 1) for demonstrating the operation control processing based on the operation command of the content of one Embodiment. It is a figure (the 1) for demonstrating the operation control processing based on the operation command of the content of one Embodiment. It is a flowchart (the 1) which shows the operation of the operation control processing based on the operation command including the operation command candidate presentation processing executed by the system control device of one Embodiment. It is a flowchart (the 1) which shows the operation of the operation control processing based on the operation command including the operation command candidate presentation processing executed by the system control device of one Embodiment.

Hereinafter, embodiments will be described. The embodiments described below do not unreasonably limit the content of the present invention described in the claims. Moreover, not all of the configurations described in the present embodiment are essential constituent requirements of the present invention. Further, in the following embodiment, an image in an object space in which a character to be viewed (hereinafter, also referred to as “target character”) is arranged for an audience housed in a given venue in the real space. Is an embodiment in which the operation control system, the content screening system, and the program according to the present invention are applied to the content screening system to be screened as the content.

[1] Content Screening System First, the outline and outline configuration of the content screening system 10 of the present embodiment will be described with reference to FIG. Note that FIG. 1 is a diagram showing an example of a system configuration showing a configuration of a network system including the content screening system 10 of the present embodiment, and FIG. 2 is a diagram showing a relationship between the content screening system 10 and a large display device 12 at an event venue. It is a figure for demonstrating the relationship between an acting character which is an example of a target character, and an audience.

The content screening system 10 of the present embodiment is, for example, a plurality of audiences housed in a given venue (hereinafter, referred to as “screening venue”) in a real space where content is screened, such as an event venue. Screening of content that visualizes the acting performance (specifically, singing live) of a group consisting of idols (so-called characters that operate in the object space (hereinafter referred to as "acting characters")). It is a screening system.

Then, the content screening system 10 of the present embodiment generates a moving image of the acting performance while controlling the movement of the acting character based on the algorithm defined in the content and the instruction of the operator, and the image. It has a configuration that excites the audience and provides a fresh performance as a moving image while improving the presence of the acting character and the live feeling in the performance of the acting character.

Specifically, as shown in FIG. 1, the content screening system 10 is connected to the server device 11 via a network such as the Internet, and the server device 11 allows another content screening system (10A, 10B, 10C). ) Is included in the management.

Then, as shown in FIGS. 1 and 2, the content screening system 10 is installed in a given venue (for example, an event venue) in the real space, and is a large display device 12 that visually displays the content by the audience 3. , A system control device 20 that accepts various operations of the operator 5 and displays contents on the large display device 12.

The large display device 12 is installed on a stage 2 provided in a venue where contents such as an event venue are screened, and images an object space including an acting character 8 that performs a performance such as a song, a dance, a play, or a lecture. It is a device to do.

The large display device 12 is composed of a large screen formed on the stage 2 and a projector that projects an image (for example, from the back) on the screen.

In addition, the large display device 12 screens live performances of acting characters 8 such as singing music, playing musical instruments, dancing, and street performance as contents.

The large display device 12 may be composed of a large liquid crystal display device, a stereoscopic imaging device such as a hologram, or the like.

The system control device 20 is composed of, for example, a tablet-type information terminal device, a personal computer, a smartphone, or an information processing device such as a game machine, and can be connected to the server device 11 via a network such as the Internet (WAN) or LAN. Device. The communication line between the system control device 20 and the server device 11 may be wired or wireless.

In addition, the system control device 20
(1) A virtual camera is set in the object space in which the acting character is arranged, and an image of the object space with the virtual camera as a viewpoint is generated.
(2) An image of the object space centered on the acting character is displayed on a large display device that is visually recognized by the audience.
(3) An operation command selected from a plurality of operation commands for controlling the movement of the acting character to be imaged during the screening of the content according to the spectator information indicating information about the spectator is used as an operation command candidate. , Present to the operator,
(4) The presented operation command candidate is specified by the operation instruction of the operator, and the specified operation command candidate is accepted as an operation command.
(5) Control the movement of the displayed acting character based on the received operation command.
It has a configuration.

The system control device 20 has a configuration in which an image with a viewpoint of a virtual camera arranged in an object space, centered on the acting character, is displayed on the large display device 12 so that the audience can view it as content.

In particular, when the system control device 20 selects one content in which the type of acting character such as gender, age, number of people, and idle character is defined, the object space is followed by the operation of the operator while following the algorithm included in the content. It has a configuration for imaging the acting performance of the acting character (for example, an idol character) arranged in.

For example, as shown in FIG. 2, the system control device 20 of the present embodiment is a group of acting characters composed of a pair of a first acting character (idle character) 8A and a second acting character (idle character) 8B. The live performance executed by is displayed on the large display device 12 as content.

The server device 11 provides various data to each content screening system 10 by using, for example, an API (application programming interface) or a predetermined platform, or acquires and manages spectator information from each content screening system 10. It is an information processing device.

In particular, the server device 11 provides each content screening system 10 with data (hereinafter, referred to as “content data”) for imaging the performance of the acting character in the content screening system 10 and outputting sound.

Further, the server device 11 has a storage area (a storage unit 140 described later), and uses the recording area to provide billing information for each spectator, action history information (hereinafter, referred to as “behavior history information”), and the like. Perform information management. However, the server device 11 may store in a database (storage device, memory in a broad sense) connected via a network (intranet or the Internet) instead of the storage area.

The server device 11 may be composed of one (device, processor) or a plurality of (device, processor).

Further, in the present embodiment, the content screening system 10 is in the object space based on the content data (specifically, the control data that defines the algorithm described later) or the operation command of the received operator. Although the operation control of the acting character is performed, the server device 11 may perform a part or all of each process of each operation control of the acting character.

In this case, for example, when all the processes are executed by the server device 11, the content screening system 10 displays display control data for displaying the operation-controlled acting characters in the object space transmitted from the server device 11. The operation information input by the operator is transmitted to the server device 11 while receiving the data such as the above and controlling the display thereof.

[2] Server Device First, the outline and configuration of the server device 11 of the present embodiment will be described with reference to FIG. Note that FIG. 3 is a diagram showing a functional block of the server device 11 of the present embodiment. Further, the server device 11 of the present embodiment may have a configuration in which some of the components (each part) of FIG. 3 are omitted.

The server device 11 mainly includes a server processing unit 100 that executes processing related to the provision of content to be provided, an input unit 120 for use by an administrator or other input, a display unit 130 that performs a predetermined display, and audience information. Includes a storage unit 140 that stores various data used for the content, an information storage medium 180 that stores predetermined information, and a communication unit 196 that communicates with the content screening system 10 and others.

The input unit 120 is used by a system administrator or the like for setting related to contents, other necessary settings, and inputting data. For example, the input unit 120 of the present embodiment is configured by a mouse, a keyboard, or the like.

The display unit 130 displays an operation screen for the system administrator. For example, the display unit 130 of the present embodiment is configured by a liquid crystal display or the like.

The information storage medium 180 (a medium that can be read by a computer) stores programs, data, and the like, and its functions are an optical disk (CD, DVD), an optical magnetic disk (MO), a magnetic disk, a hard disk, and a magnetic tape. , Or a memory (ROM) or the like.

The communication unit 196 performs various controls for communicating with the outside (for example, a terminal, another server, or another network system), and its function is hardware such as various processors or ASICs for communication. It is composed of hardware and programs.

The storage unit 140 serves as a work area for the server processing unit 100, the communication unit 196, and the like, and its function is configured by a RAM (VRAM) or the like. The information stored in the storage unit 140 may be managed in a database.

Further, in the present embodiment, the storage unit 140 includes a main storage unit 142 used as a work area, content data 144 indicating data related to the content provided to each content screening system 10, and an audience indicating audience information regarding the audience. Information 146 and data collected for each screened content (hereinafter referred to as "content collection data") 148 are stored. However, in addition to the above-mentioned information, various information necessary for content calculation is stored in the storage unit 140.

The content data 144 includes data (hereinafter, also referred to as “control data”) in which an algorithm used in each content screening system 10 is defined in association with the content ID for each content, and the acting character performs acting performance. It includes various sound data such as music and dialogue when executing, and various image data used when imaging an object space including acting characters.

In particular, the algorithm includes the movement of the acting character in the object space, the position and angle of the virtual camera set in the object space, the timing of presenting the operation command, the timing of controlling by the operation command, and the type of the operation command to be presented. , The flow of the screening of the content indicating the output timing of various sounds such as music and dialogue when the acting character executes the acting performance is stipulated.

Further, in the content data 144, the type, timing, and number of times of each operation command actually used by each content screening system 10 are stored as usage history information for each content.

The spectator information 146 is associated with the spectator ID for each spectator.
(1) Audience attribute information such as gender, age, and residential area,
(2) Audience preference information such as favorite acting characters,
(3) Information on terminal devices such as smartphones and ID tags owned by the spectator (for example, information indicating the position (seat) at the time of viewing such as information identifying the spectator's individual and ticket information),
(4) Information that is stored in association with each spectator such as an ID or in association with a terminal device owned by the spectator, and the target character being viewed or the target character is performing. Information (game play information) regarding the play status and history (including billing history) of a predetermined game such as a game related to an event,
(5) Information stored using a terminal device owned by the spectator, which is related to the target character, action history information indicating participation in an event in which the target character is performing, and the event or performance. Billing information and billing history information from the designated online shopping site and actual sales stores (including inside the event venue), as well as
(6) Biological information (for example, face image, voice information (including voice print information of each spectator and frequency characteristic information indicating frequency characteristics), posture information, calorific value information, and a predetermined device (smartphone, etc.) from each spectator. Information similar to biological information such as light information using psyllium)) is included.

In addition, as described above, the billing history includes the billing history in the game and the billing related to the purchase of services and products, the billing date and time indicating the date and time when these billing was executed, the purchase of items and the purchase of twists. The type of billing element such as, the billing amount indicating the billed amount, etc. are included.

The content collection data 148 includes, for each content, the screening record (date and time and number of times) of the content screened in each content screening system 10, the data of the operation command received by the operator's instruction when the content was screened, and the data of the operation command. Data indicating the content of motion control of each acting character and data are stored.

The server processing unit 100 performs various processes using the main storage unit 142 in the storage unit 140 as a work area. The functions of the server processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs.

The server processing unit 100 performs various processes of the present embodiment based on the program (data) stored in the information storage medium 180. That is, the information storage medium 180 stores a program for operating the computer as each part of the present embodiment (a program for causing the computer to execute the processing of each part).

For example, the server processing unit 100 (processor) controls the entire server device 11 based on the program stored in the information storage medium 180, and provides the content data to the content screening system 10. Each process is performed, such as a process of providing spectator information in conjunction with the content screening system 10 and a process of registering operation commands and spectator information collected at the time of screening of the content transmitted from the content screening system 10 in the storage unit 140.

Specifically, the server processing unit 100 of the present embodiment has at least a communication control unit 101, a content management unit 102, and an audience management unit 103.

The communication control unit 101 performs a process of transmitting and receiving data via the content screening system 10 and the network. That is, the server device 11 transmits the content data to the content screening system 10 by the communication control unit 101, and receives various information such as spectator information and operation commands transmitted from the content screening system 10.

The content management unit 102 registers data such as the screening results of the content screened in each content screening system 10 and the operation commands used during the screening of each content screening system 10 in the content collection data 148.

In addition, the content management unit 102 analyzes each content collection data of the same content at a predetermined timing, optimizes the operation command presented to the operator, and sets the operation command as the default during the screening of the content. Updates the algorithm in which is set.

The spectator management unit 103 performs processing related to the registration procedure of each spectator (registration of attribute information, preference information, etc.) and storage management of data associated with the account (audience ID).

Specifically, the spectator management unit 103 controls the assignment of accounts for registering and managing each spectator information, and spectator information of each spectator detected during content execution (hereinafter, also referred to as "detected spectator information", for example, In addition to the biometric information of each spectator in the face image, it also includes operation commands and performance timing when the biometric information is detected) and personal information such as attribute information and preference information related to the spectator. Management, virtual account setting and deposit / withdrawal control that is linked to the account and used as a source of payment for the execution of the billing element, registration of the billing history related to the billing element by the spectator of the account, and registration of the game history Etc. are executed.

In particular, the spectator management unit 103 stores and manages the detected spectator information by associating each information detected by the content screening system 10 with the spectator information account.

In addition, the spectator management unit 103 includes game play information, billing history information, and other information on a game server device (not shown), a management server device that operates a mail-order site, or another server (not shown) such as a credit card or a bank payment server device. By interlocking with the device and exchanging predetermined data, each information is registered in the audience information 146.

[3] System Control Device Next, the content screening system 10 of the present embodiment will be described with reference to FIG. Note that FIG. 4 is a diagram showing a functional block of the content screening system 10 of the present embodiment. Further, the content screening system 10 of the present embodiment may have a configuration in which some of the constituent elements (each part) of FIG. 4 are omitted.

The spectator information detection unit 250 detects various ecological information or information equivalent to the ecological information such as an image pickup camera, an infrared sensor, a sound collecting microphone, a seating sensor, thermography, or a marker unit possessed by each spectator and its detection device (camera). It has a function to do.

Then, the spectator information detection unit 250 detects predetermined biometric information or the like according to an algorithm during the screening of the content, and provides the operation command candidate presentation processing unit 214 with the detected information.

For example, the spectator information detection unit 250 of the present embodiment constitutes the detection means of the present invention.

The input unit 260 is for the operator to input an operation command, and its function can be realized by a touch panel, a touch panel type display, a trackball, a keyboard, a mouse, or the like.

That is, when the input unit 260 is configured by a device that executes touch operation input such as a touch panel, the input unit 260 can detect two-dimensional indicated position coordinates (x, y) on the screen on which the image is displayed. The detection unit 262 is provided.

For example, the input unit 260 includes a detection unit 262 capable of detecting two-dimensional contact position coordinates (x, y) in the contact detection area (touch panel).

In the present embodiment, the contact position is position information obtained from the contact detection region by the operator's contact operation (including direct contact and indirect contact by a pointing device).

The operator's contact operation according to the present embodiment includes a direct touch operation on the monitor display unit 290 (touch panel) by the user's finger and an indirect operation performed via a device such as a pointing device. In addition to the touch operation on the monitor display unit 290 (touch panel), the monitor display unit using an optical beam or a camera without directly or indirectly touching or sliding the monitor display unit 290 (touch panel). A pseudo touch operation on the 290 (touch panel) is also included.

In the present embodiment, the monitor display unit (display) 290 for the operator is a touch panel type display in which a liquid crystal display and a touch panel for detecting the contact position of the operator are laminated. That is, in the present embodiment, the monitor display unit 290 functions as the input unit 260.

Further, the contact operation with the monitor display unit 290 may be performed using a fingertip or an input device such as a touch pen.

Further, the input unit 260 may include a button, a lever, a steering wheel, a microphone, an acceleration sensor, and the like capable of inputting operation information (operation signal) other than the indicated position.

The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, and the like, and its function can be realized by a RAM (VRAM) or the like. The storage unit 270 of the present embodiment includes a main storage unit 271 used as a work area, an image buffer 272 in which the final display image and the like are stored, audience information 273 indicating information about the audience, and an algorithm. It includes a content data storage unit 274 that stores various data necessary for executing the screening of contents such as data, operation command data, and table data.

The spectator information 273 stores predetermined spectator information such as spectator information of each spectator detected by the spectator information detection unit 250 and attribute information of spectators visiting the screening venue.

During the content screening, the content data storage unit 274 stores the type, timing, and number of times of each operation command used in the screening of the content as usage history information.

A part of these may be omitted, or a part of the storage unit 140 of the server device 11 may be configured. Further, for example, the storage unit 270 of the present embodiment constitutes the storage unit of the present invention alone or together with the storage unit 140 of the server device 11.

In the spectator information 273, for example, in the spectator information stored in the storage unit 140 of the server device 11, the spectator information of the spectator who is scheduled to watch the performance to be screened or is being watched is stored.

The content data storage unit 274 stores various information necessary for screening the content, information such as operation commands used during the screening of the content, and condition information used in various determination processes. ..

The information storage medium 280 (a medium that can be read by a computer) stores programs, data, and the like, and its functions are an optical disk (CD, DVD), an optical magnetic disk (MO), a magnetic disk, a hard disk, and a magnetic tape. Alternatively, it can be realized by a memory (ROM) or the like.

Further, the information storage medium 280 can store a program for operating the computer as each part of the present embodiment (a program for causing the computer to execute the processing of each part). As will be described later, the processing unit 200 performs various processes of the present embodiment based on the program (data) stored in the information storage medium 280.

The monitor display unit 290 outputs an image generated by the present embodiment, and its function is a projection device composed of a CRT, an LCD, a touch panel type display, an HMD (head mounted display), for example, a screen and a projector. It can be realized by such as.

In particular, in the present embodiment, the monitor display unit 290 also functions as an input unit 260 in which the operator inputs various inputs by using the touch panel display as described above. Here, as the touch panel, for example, a touch panel such as a resistive film type (4-wire type, 5-wire type), a capacitance method, an electromagnetic induction method, an ultrasonic surface acoustic wave method, or an infrared scanning method can be used.

The sound output unit 292 outputs the sound generated by the present embodiment, and its function can be realized by a speaker, headphones, or the like.

The communication unit 296 performs various controls for communicating with an external device (for example, a host device or another terminal device), and its functions include hardware such as various processors or communication ASICs, and a program. It can be realized by such as.

The system control device 20 receives and receives programs and data for operating the computer as each part of the present embodiment stored in the information storage medium of the server device 11 and the storage unit 270 via the network. The program or data may be stored in the information storage medium 280 or the storage unit 270. The case where the system control device 20 functions by receiving a program or data in this way can also be included in the scope of the present invention.

The processing unit 200 (processor) presents an operation command based on the input data or the program from the input unit 260, according to the program and algorithm data provided by the server device 11, or in conjunction with the server device 11. Processing, content screening processing including operation command reception processing, image generation processing, sound generation processing, and the like are performed.

In particular, in the present embodiment, the content screening process includes a process of starting the screening of the content when the screening start condition of the content is satisfied, and a process of arranging various objects used for the screening of the content such as acting characters. , A process of controlling the movement of the acting character according to an algorithm or an operation command of the operator, a process of ending the game when the screening end condition of the content is satisfied, and the like.

Further, the processing unit 200 performs various processing using the storage unit 270 as a work area. The function of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), or a program.

In particular, the processing unit 200 of the present embodiment includes an object space setting unit 210, a touch detection processing unit 211, a content display control unit 212, a monitor display control unit 213, an operation command candidate presentation processing unit 214, and a movement processing unit. 215, a virtual camera control unit 216, a content editing processing unit 217, a content screening execution control unit 218, a content management unit 219, a drawing unit 220, and a sound processing unit 230 are included. Further, a configuration in which some of these may be omitted may be used.

For example, the touch detection processing unit 211 of the present embodiment constitutes the receptionist of the present invention, and the content display control unit 212 constitutes the audience display control means of the present invention. Further, for example, the monitor display control unit 213 of the present embodiment constitutes the monitor display control means of the present invention, and the operation command candidate presentation processing unit 214 constitutes the presentation means of the present invention. Further, for example, the content editing processing unit 217 of the present embodiment constitutes an editing means, and the content screening execution control unit 218 constitutes the registration means and the operation control means of the present invention.

The object space setting unit 210 is a character object (acting character), a movement path, a building, a tree, a pillar, a wall, a map (topography), and various other objects (sprites, billboards, polygons, free curved surfaces, or subdivisions). Performs a process of arranging and setting an object composed of primitive surfaces such as a surface in an object space (that is, a game space).

Specifically, the object space setting unit 210 determines the position and rotation angle (synonymous with orientation and direction) of the object (model object), and rotates the object (model object) at the position (X, Y) or (X, Y, Z). Place the object by an angle (rotation angle around the X, Y axis) or (rotation angle around the X, Y, Z axis).

The object space includes both a so-called virtual two-dimensional space and a virtual three-dimensional space. The two-dimensional space is, for example, a space in which objects are arranged in two-dimensional coordinates (X, Y), and the three-dimensional space is, for example, a space in which objects are arranged in three-dimensional coordinates (X, Y, Z). be.

Then, when the object space is a two-dimensional space, the objects are arranged based on the priority set for each of the plurality of objects. For example, it is possible to arrange objects in order from the object (sprite) that you want to appear to be on the back side, and to stack the objects that you want to appear to be on the front side.

Also, if an object with a large drawing size is placed below the image and an object with a small drawing size is placed above the image, the object space corresponding to the upper part of the screen can be made to appear to be on the back side. It is possible to make it appear that the object space corresponding to the lower part of is on the front side.

On the other hand, when the object space is a three-dimensional space, the objects are arranged in the world coordinate system.

The touch detection processing unit 211 receives and recognizes instruction information for executing operation control of the acting character such as an operation command input by the operator to the input unit 260.

Specifically, the touch detection processing unit 211 acquires the contact position (two-dimensional contact position coordinates) in the contact detection area (touch panel) for detecting the contact operation (that is, the touch operation) of the operator as the designated position.

In particular, the touch detection processing unit 211
(1) Change from a state in which no touch operation is input to the screen (hereinafter, referred to as "non-contact state") to a state in which touch operation is input (hereinafter, referred to as "contact state") (hereinafter, "state change"). ”) Is detected, and the detection position where the state change is executed is detected.
(2) Detection of the operation command candidate displayed at the detection position as an operation command, an instruction to edit the operation command (for example, addition, change, and deletion of the default operation control in the performance character for the operation command), or performance. Performs viewing detection for a particular audience who enjoys the character's special effects.

The content display control unit 212 executes various controls on the large display device 12 to display an image of the object space, centered on the acting character, so that the audience can view it.

The monitor display control unit 213 is related to operation support such as displaying the content centered on the acting character being screened on the monitor display unit 290 and displaying operation command candidates to the operator during the screening of the content. Various displays are displayed on the monitor display unit 290.

The movement processing unit 215 performs an object movement operation in the object space. That is, the movement processing unit 215 moves the moving object in the object space based on the input data or program (movement algorithm) or various data (motion data) input by the operator by the input unit 260, or Performs processing to control the movement (motion, animation) of a moving object.

Specifically, the movement processing unit 215 of the present embodiment includes movement information (movement direction, movement amount, movement speed, position, rotation angle, or acceleration) and motion information (position or rotation angle of each part object) of the object. ) Is sequentially obtained for each frame. Here, the frame is a unit of time for performing object movement processing, motion processing (simulation processing), and image generation processing. Then, in the present embodiment, the frame rate may be fixed or variable depending on the processing load.

The virtual camera control unit 216 generates an image that is visible from a given viewpoint and that appears to have depth. In this case, the virtual camera control unit 216 performs control processing of the virtual camera (viewpoint) for generating an image that can be seen from a given (arbitrary) viewpoint in the object space. Specifically, processing (processing for controlling the viewpoint position and line-of-sight direction) for controlling the position (X, Y, Z) or rotation angle (rotation angle around the X, Y, Z axes) of the virtual camera is performed.

For example, when an object (for example, a character, a ball, a car) is photographed from behind by a virtual camera, the position or rotation angle of the virtual camera (virtual camera) so that the virtual camera follows a change in the position or rotation of the object. Orientation) is controlled.

In this case, the virtual camera can be controlled based on the information such as the position, rotation angle, and speed of the object obtained by the movement processing unit 215. Alternatively, the virtual camera may be rotated at a predetermined rotation angle, or may be controlled to move in a predetermined movement path. Further, in this case, the virtual camera is controlled based on the virtual camera data for specifying the position (movement path) or the rotation angle of the virtual camera.

When there are a plurality of virtual cameras (viewpoints), the above control processing is performed for each virtual camera.

The content editing processing unit 217 executes an editing process for editing the operation control of the target character corresponding to the operation command based on the instruction of the operator during the screening of the content or at other timings.

The content screening execution control unit 218
(1) Execute an algorithm control process that controls the operation of the target character to be imaged according to the algorithm defined in the control data included in the content data.
(2) When an operation command is received during the algorithm control process, it is linked with the action of the acting character that is the target of the algorithm control process, or instead of the action of the acting character specified in the algorithm control process. Then, the operation of the indicated acting character based on the received operation command is controlled.

Further, the content screening execution control unit 218 provides the server device 11 with the operation command used and the operation of the acting character based on the operation command, the detected detected spectator information and the used spectator information as content collection data during the content screening. do.

Specifically, the content screening execution control unit 218 uses the operation command received during the screening and the detected audience information at a predetermined timing after the content screening as information for analyzing the content screening result information, or. , Registered in the server device 11 as usage history information used for other screenings of the same content.

The content management unit 219 uses the type and timing of the operation command received during the screening of the content and the spectator information detected when the action of the acting character based on the operation command is executed as the content screening result information. , Stored in the content data storage unit 274.

In addition, the content management unit 219 uses the operation command received during the screening and the detected audience information at a predetermined timing after the content is screened as information for analyzing the content screening result information, or in the same content. It is registered in the server device 11 as usage history information used for other screenings.

The drawing unit 220 performs drawing processing based on the results of various processing (each processing related to content screening) performed by the processing unit 200, thereby generating an image, and the large display device 12 and the monitor display unit (display). Output to 290.

Further, the image generated by the drawing unit 220 may be a so-called two-dimensional image or a so-called three-dimensional image, and the drawing unit 220 is an image that can be seen from a virtual camera in the object space and is a screen. Generate the image to be displayed above.

Here, when generating a two-dimensional image, the drawing unit 220 draws in order from the set low-priority objects, and when the objects overlap, the high-priority objects are overwritten and drawn. ..

When generating a three-dimensional image, the drawing unit 220 of the present embodiment first uses the vertex data (positional coordinates of the vertex, texture coordinates, color data, normal vector or α value) of each vertex of the object (model). Object data (model data) including (etc.) is input, and vertex processing is performed based on the vertex data included in the input object data. In addition, when performing the vertex processing, the vertex generation processing (tessellation, curved surface division, polygon division) for subdividing the polygon may be performed as necessary.

In the vertex processing, geometry processing such as vertex movement processing, coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping processing, perspective conversion, or light source processing is performed, and the object is processed based on the processing result. Change (update, adjust) the given vertex data for the constituent vertex group. Then, rasterization (scanning conversion) is performed based on the vertex data after the vertex processing, and the faces of the polygons (primitives) are associated with the pixels. Then, following the rasterization, pixel processing (fragment processing) for drawing the pixels (fragments constituting the display screen) constituting the image is performed.

In pixel processing, various processes such as texture reading (texture mapping), color data setting / change, translucent composition, and antialiasing are performed to determine the final drawing color of the pixels that make up the image, and perspective conversion is performed. The drawing color of the object is output (drawn) to the image buffer 272 (frame buffer, a buffer that can store image information in pixel units. VRAM, rendering target). That is, in the pixel processing, per pixel processing is performed in which image information (color, normal, brightness, α value, etc.) is set or changed in pixel units.

As a result, an image that can be seen from a virtual camera (given viewpoint) set in the object space is generated. When there are a plurality of virtual cameras (viewpoints), an image can be generated so that the image seen from each virtual camera can be displayed as a divided image on one screen.

The vertex processing and pixel processing performed by the drawing unit 220 are performed by hardware that enables programming of polygon (primitive) drawing processing by a shader program written in a shading language, so-called programmable shader (vertice shader or pixel shader). It may be realized. In the programmable shader, the processing in units of vertices and the processing in units of pixels can be programmed, so that the degree of freedom in the drawing processing content is high, and the expressive power can be greatly improved compared to the fixed drawing processing by hardware. ..
Then, when drawing the object, the drawing unit 220 performs geometry processing, texture mapping, hidden surface erasing processing, α blending, and the like.

In the geometry processing, the object is subjected to processing such as coordinate transformation, clipping processing, perspective projection conversion, or light source calculation. Then, the object data after the geometry processing (after the perspective projection conversion) (position coordinates of the vertices of the object, texture coordinates, color data (luminance data), normal vector, α value, etc.) is stored in the storage unit 270.

In the texture mapping, a process of mapping the texture (texel value) stored in the texture storage unit of the storage unit 270 to the object is performed. Specifically, the texture (surface properties such as color (RGB) and α value) is read from the texture storage unit of the storage unit 270 using the texture coordinates and the like set (assigned) to the vertices of the object, and a two-dimensional image is obtained. Map the texture that is to the object. In this case, processing for associating pixels with texels and bilinear interpolation as texel interpolation are performed.

In this embodiment, when drawing an object, a process of mapping a given texture may be performed. In this case, the color distribution (texel pattern) of the mapped texture can be dynamically changed.

Further, in this case, textures having different color distributions (pixel patterns) may be dynamically generated, or a plurality of textures having different color distributions may be prepared in advance so that the textures to be used are dynamically switched. You may. Further, the color distribution of the texture may be changed for each object.

In the hidden surface erasing process, the hidden surface erasing process is performed by the Z buffer method (depth comparison method, Z test) using the Z buffer (depth buffer) in which the Z value (depth information) of the drawing pixel is stored. That is, when drawing the drawing pixel corresponding to the primitive of the object, the Z value stored in the Z buffer is referred to, and the Z value of the referenced Z buffer and the Z value of the drawing pixel of the primitive are referred to. In comparison, if the Z value in the drawing pixel is the Z value (for example, a small Z value) that is on the front side when viewed from the virtual camera, the drawing pixel is drawn and the Z value in the Z buffer is newly added. Update to a Z value.

In α blending (α synthesis), the drawing unit 220 performs a translucent synthesis process (usually α blending, addition α blending, subtraction α blending, etc.) based on the α value (A value). The α value is information that can be stored in association with each pixel (texel, dot), and is, for example, plus alpha information other than color information. The α value can be used as mask information, semi-transparency (equivalent to transparency and opacity), bump information, and the like.

In particular, the drawing unit 220 in the present embodiment may generate an image to be displayed in the display area corresponding to the contact detection area. The image to be displayed in the display area may be, for example, an image including an object.

The sound processing unit 230 performs sound processing based on the results of various processing performed by the processing unit 200, and generates various sounds when the acting character such as BGM, sound effect, or voice performs a performance. It is output to the sound output unit 292.

The timer unit 240 uses the system clock to perform timekeeping for executing a process of measuring a timer value such as the current date and time, a time limit, and time management related to the screening of content based on an algorithm.

[4] Outline of the method of the present embodiment [4.1] Outline Next, an outline of the method of the present embodiment (operation command presentation processing) will be described.

The content screening system 10 of the present embodiment has a configuration in which the acting character can be easily operated by the operator without requiring special skills when the content is screened.

Then, the content screening system 10 of the present embodiment can operate the character based on a simple operation according to the atmosphere of the spectator viewing the acting character and the state of the spectator, and the target character to be imaged. It has a configuration that can excite the audience and realize a fresh performance while improving the sense of existence and the live feeling in the performance of the target character.

In particular, the content screening system 10
(1) While arranging the acting character in the object space according to the algorithm included in the content data acquired from the server device 11, the operation of the acting character is controlled, and (2) the operation is virtually controlled centering on the acting character. The image of the object space with the viewpoint of the camera is displayed on the large display device 12 and the monitor display unit 290.
It has a configuration in which each process is executed as a content screening process for screening the content.

Then, the content screening system 10 is in the process of executing the content screening process.
(1) Operation command candidates that are presented to the operator as operation command candidates, which are selected by the operator according to the spectator information from among a plurality of operation commands for controlling the operation of the target character in the object space. Presentation processing and
(2) When one or more operation command candidates are specified from the operation command candidates presented by the operator, the specified operation command candidate is accepted as an operation command, and the performance is performed based on the accepted operation command. Processing that controls character movements (hereinafter, also referred to as "motion control processing based on operation commands"),
It has a configuration to execute each process of.

By having such a configuration, the content screening system 10 causes the target character to excite the audience when it is determined by the audience information that the audience who is viewing the target character such as the event venue is not excited. Audience observing the target character, such as making it possible to select a command to execute MC (line or microphone performance), or to make it possible to select an operation command to execute a violent dance by the target character. The operator can select the action of the target character according to the situation (that is, the current state of the spectator).

Further, the content screening system 10 controls the movement of the target character for women when it is determined from the audience information that there are many women in the audience viewing the target character, or the content screening system 10 controls the operation of the target character for women, or a predetermined goods in the past behavior. If it is determined that the target character has been purchased, an operation command that controls the behavior of the target character according to the behavior history and attributes of the spectator who is observing by causing the spectator to perform a special effect of the target character. Can be selected by the operator.

As described above, the content screening system 10 of the present embodiment can operate the acting character according to the atmosphere of the spectator and the state of the spectator while simplifying the operation by the operator, so that the acting character to be imaged can be operated. While improving the sense of existence and the live feeling in the performance of the acting character, it is possible to excite the audience and realize a fresh performance.

Therefore, the content screening system 10 can control the movement (that is, performance) of the target character, which is highly satisfying to the audience.

[4.2] Operation Command Candidate Presentation Process Next, the operation command candidate presentation process of the content of the present embodiment will be described with reference to FIGS. 5 to 7. It should be noted that FIGS. 5 to 7 are diagrams for explaining the operation command candidate presenting process of the content of the present embodiment.

(Basic principle)
The operation command candidate presentation processing unit 214 controls the operation of the acting character at a predetermined timing specified in the algorithm from the start to the end of the performance of the acting character in the operation command candidate presentation processing unit 214. To present.

In particular, the operation command candidate presentation processing unit 214
(1) First operation command candidate that presents an operation command selected by the operator's intention as an operation command candidate from among a plurality of operation commands for controlling the operation of the target character in the object space while referring to the spectator information. Presentation processing and
(2) An operation command extracted from a plurality of operation commands for controlling the operation of the target character in the object space according to the spectator information and selected by the operator is operated as an operation command candidate. The second operation command candidate presentation process to be presented to the person is executed.

In the present embodiment, the operation command candidate presentation processing unit 214 executes either the first operation command candidate presentation process or the second operation command candidate presentation process during the performance of the acting character based on the algorithm. do.

(First operation command candidate presentation processing)
The operation command candidate presentation processing unit 214 interlocks with the server device 11, and performs the acting performance of the acting character while identifying in advance the spectators who are coming to the event venue by using the admission ticket, for example, by associating the ticket ID with the spectator ID. Audience information such as attributes, preference information, game play information, and billing history information of the spectators who are viewing is acquired from the server device 11.

Then, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213, and at the timing specified by the algorithm, the acquired spectator information, the analysis information showing the result of analyzing the characteristics of the spectator by the spectator information, or the analysis information. , Both information is displayed on the monitor display unit 290 and presented to the operator.

In particular, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213 to acquire an operation command for the acting character to easily select an operation command that matches the audience. The spectator information is displayed in a list on the monitor display unit 290.

Further, when the operation command candidate presentation processing unit 214 can specify a specific spectator in the event venue (specifically, the position of the specific spectator in the venue) by a seat or the like, the operation command candidate presentation processing unit 214 is a monitor display control unit. In conjunction with 213, the acquired spectator information is associated with the spectator (or the position of the spectator) and displayed in a list on the monitor display unit 290.

For example, as shown in FIG. 5A, the operation command candidate presentation processing unit 214 causes the operator to select an operation command for executing an appropriate effect when performing an effect on the front row of the event venue. In addition to the operation command candidates defined at the timing by the algorithm, the spectator information of the spectators in the front row of the event venue is displayed in a list on the monitor display unit 290.

Note that FIG. 5A shows an example in which each operation command of commands A to D formed on the first acting character 8A is displayed on the monitor display unit 290 of the touch panel as an operation command candidate together with the audience information. show.

In particular, in FIG. 5A, in addition to the above, image parts for supporting operator operations such as command execution speed and content screening time that can be changed by touch operation are displayed on the monitor display unit 290. An example is shown.

Further, as shown in FIG. 5B, when the content screening execution control unit 218 receives an operation command in which the acting character operates from the operation command candidates presented by the touch detection processing unit 211, the content screening execution control unit 218 receives the operation command. Based on this, the movement control of the target acting character is executed.

Then, in FIG. 5B, since there is a male spectator who likes the first acting character 8A and has a charge amount of a certain amount or more in the lower front row of the stage 2, there are spectators among the operation command candidates. An example is shown in the case where the first acting character 8A receives the operation command A of the performance "waving poorly" to the lower front row of the stage 2. However, the operation control of the acting character in the content screening execution control unit 218 will be described later.

On the other hand, the operation command candidate presentation processing unit 214 sets the characteristics of the entire spectator and each spectator based on the spectator information of the spectators who are visiting the event venue before or during the performance of the acting character. Characteristics (individual characteristics) or both may be analyzed.

For example, the operation command candidate presentation processing unit 214 has a characteristic of the entire audience.
(A1) Audience composition ratio (age, gender difference, regional difference),
(A2) Approval rating of each acting character as a favorite character (many spectators like character A, etc.),
(A3) The total billing amount and average billing amount of the entire audience, and
(A4) Analyze the average play time in a specific game.
Further, for example, the operation command candidate presentation processing unit 214 has a characteristic of each spectator.
(B1) Audiences who are charging a certain amount or more,
(B2) Audiences who have purchased a specific product or service, or
(B3) Predetermined conditions in a specific game, such as possession of a specific player character or item in a specific game, being at a predetermined level or higher, or having a predetermined point or experience value. Audience,
To analyze.

Then, the operation command candidate presentation processing unit 214 presents the operation command as an operation command that can be selected by the operator's intention while referring to the spectator information. Therefore, the operation command candidate presentation processing unit 214 is linked with the monitor display control unit 213 and presented by an algorithm. The above analysis result is displayed at a position on the monitor display unit 290 that can be visually recognized by the operator at the same time as the operation command specified to be performed.

For example, as shown in FIG. 6A, the operation command candidate presentation processing unit 214 indicates that there are a certain percentage or more of spectators who have visited the event more than a predetermined number of times (for example, three times). When analyzed, in order to make it easy for the operator to select an operation command to execute a special effect such as throwing a kiss or issuing a line to the spectator, a charge history of a certain amount or more is set for the spectator. The presence and the seat position of the corresponding spectator are displayed in a list on the monitor display unit 290 together with the operation command candidates.

In such a case, the content screening execution control unit 218 executes, for example, the operation control of the acting character singing a new song that has not been shown so far, as a special effect when the number of visits is large.

Further, for example, when the operation command candidate presentation processing unit 214 analyzes that a certain percentage or more (for example, 70%) of female spectators are present at the event venue, as shown in FIG. 6B, a female spectator is present. In order for the operator to easily select an operation command for executing the effect, the monitor display unit 290 displays a display indicating that the female ratio of the spectator is above a certain level.

In such a case, the content screening execution control unit 218 executes, for example, the operation control of the acting character that emits the dialogue for women.

On the other hand, as shown in FIGS. 5 and 6, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213 and displays an image of the acting character to be operated on the monitor display unit 290 as a touch panel. , Each operation command candidate may be displayed so that the operation command can be selected and operated on the image of the acting character.

That is, in the present embodiment, by having such a configuration, the operator can intuitively execute the operation instruction of the operation command, so that the operation of the operator is simplified and the operability is improved. You can do it.

(Second operation command candidate presentation processing)
The operation command candidate presentation processing unit 214 interlocks with the spectator information detection unit 250, and at the timing specified by the algorithm, the biological information such as the spectator's facial expression and voice, the spectator's posture, the heat, or the light emitted by the spectator (for example,). Smartphones and psylliums) or information on terminal devices owned by spectators (for example, information for identifying the viewing (seat) position and individual of each spectator, game history information, purchase history information of products, etc.) Detects information similar to biological information.

That is, the operation command candidate presentation processing unit 214 is, for example, whether or not the audience is excited, whether or not the audience is enjoying, the male-female ratio (whether there are many males or many females), or, for example, the acting character. Audiences emitting red light (indicating the elements of rocking depending on the type of light, or "YES" for red light, "NO" for blue light, etc.) used in response to dialogue and games Analyze the state of the audience, such as whether there are many.

Camera In particular, the spectator information detection unit 250 includes an image pickup camera, an infrared sensor, a sound collecting microphone, a seating sensor, thermography, a marker unit possessed by each spectator and its detection device (camera) terminal device (for example, a smartphone or an ID tag). It has a function for detecting various ecological information such as a wireless receiver that acquires information transmitted from the camera, or a camera that captures the entire screening venue or acting characters.

Further, in the operation command candidate presentation processing unit 214, each spectator stands by the seating sensor according to the sound component (frequency characteristic) of each spectator collected by the sound collecting microphone according to the face image of each spectator captured by the imaging camera. Whether or not the person is excited by the amount of heat of each spectator or whether or not he / she is seated is analyzed as the state of the spectator.

Then, the operation command candidate presentation processing unit 214 has an operation command for excitement, an operation command when there are many women, or an operation command when the audience is enjoying it, from among the operation commands defined at the corresponding timing by the algorithm. An operation command or the like is selected (extracted) as an operation command candidate, and the selected operation command candidate is displayed on the monitor display unit 290 in conjunction with the monitor display control unit 213 and presented to the operator.

For example, as shown in FIG. 7, the operation command candidate presentation processing unit 214 detects the audience information when the acting character asks (that is, calls) "I'm fine!" To the audience according to the algorithm. When the unit 250 detects a cheer as a reaction (that is, a response) of a predetermined decibel (for example, 80 dB at the center of the stage) or more, the first acting character A is "Thank you everyone" from the plurality of operation commands. Operation command A that emits the line of "I'm going", operation command B that the first acting character A emits the line "Still more!" Is displayed on the monitor display unit 290 as an operation command candidate and presented to the operator.

The spectator information detection unit 250 registers the detected spectator information indicating the state of the spectator in the content data storage unit 274 as usage history information described later together with the information of the operation command.

[4.3] Operation control processing based on operation commands Next, the operation control processing based on the co-operation commands of the present embodiment will be described with reference to FIGS. 8 and 9. 8 and 9 are diagrams for explaining the operation control process based on the operation command of the content of the present embodiment.

The content screening execution control unit 218
(1) Execute an algorithm control process that controls the operation of the target character to be imaged according to the algorithm defined in the control data included in the content data.
(2) When an operation command is received during the algorithm control process, it is linked with the action of the acting character that is the target of the algorithm control process, or instead of the action of the acting character specified in the algorithm control process. Then, the operation of the indicated acting character based on the received operation command is controlled.

That is, when the content screening execution control unit 218 accepts the operation command candidate specified by the touch detection processing unit 211 among the presented operation command candidates as an operation command, the content screening execution control unit 218 controls the operation of the acting character by an algorithm (hereinafter,). , "Algorithm control processing"), or instead of the operation of the acting character in the algorithm control processing, an operation command that controls the operation of the acting character based on the received operation command. Executes the based operation control process.

In particular, the content screening execution control unit 218 changes the algorithm in the operation of the corresponding acting character according to the received operation command, or replaces it with another algorithm, thereby performing the operation of the acting character in the algorithm control process. In conjunction with each other, or instead of the operation of the acting character in the algorithm control process, the operation of the acting character based on the received operation command is controlled.

For example, it is assumed that a line is emitted while controlling the movement of the acting character's body by a specified algorithm.

In this case, when changing the algorithm, as shown in FIG. 8A, when the touch detection processing unit 211 receives the operation command of the command A "line A (exciting!)", The content As shown in FIGS. 8B and 8C, the screening execution control unit 218 changes the dialogue emitted from the acting character while controlling the movement of the body of the corresponding acting character by the algorithm.

Note that FIG. 8B shows an example in which the dialogue is changed from "energetic" to "exciting!" While controlling the movement of raising one hand toward the audience by an algorithm, for example. (C) shows an example in which the movement of raising one hand toward the audience is controlled by the algorithm, and the dialogue "Genki" defined by the algorithm is emitted.

Further, as shown in FIG. 8C, the content screening execution control unit 218 receives an operation command by the touch detection processing unit 211 by a predetermined timing (for example, the timing of starting the operation based on the operation command). If not, the operation control that emits the line "Genki" specified in the algorithm is executed.

On the other hand, in the same case, when the algorithm is replaced, as shown in FIG. 9A, when the operation command of command B is received by the touch detection processing unit 211, the content screening execution control unit 218 displays FIG. As shown in (B) and (C), the algorithm in the operation of the corresponding acting character is changed according to the received operation command.

In addition, in FIG. 9B, for example, the movement of raising one hand toward the audience by the algorithm is changed to the movement of moving while raising both hands (for example, the movement of running), and the dialogue is changed from "Genki" to "Genki". An example of changing to "Exciting!" Is shown.

Further, FIG. 9 (C) is an example in which, as in FIG. 8 (C), the movement of raising one hand toward the audience is controlled by the algorithm, and the dialogue "Genki" defined by the algorithm is emitted. show.

[4.4] Operation command candidate presentation process based on the operation command usage history Next, the operation command candidate presentation process based on the operation command usage history of the present embodiment will be described.

(Operation command candidate presentation processing based on the usage history of the operation command being shown)
The content screening execution control unit 218 registers and stores the operation command instructed by the operator (that is, the operation command received by the operation command candidate presentation processing unit 214) in the content data storage unit 274 during the content screening. ..

Specifically, the content screening execution control unit 218 registers and stores various information such as the type, timing, and number of times of operation commands in the content data storage unit 274 as usage history information.

Further, when the content screening execution control unit 218 detects the spectator information, the content screening execution control unit 218 interlocks with the spectator information detection unit 250 and uses the spectator information indicating the detected state of the spectator together with the information of the operation command, which will be described later. As history information, it is registered and stored in the content data storage unit 274.

Then, when the operation command candidate presentation processing unit 214 presents the operation command candidate, the operator uses the operation command selected according to the spectator information and the registered usage history information by the operator as the operation command candidate. Executes the operation command candidate presentation process to be presented in.

In particular, the operation command candidate presentation processing unit 214 performs, as the first operation command candidate presentation processing, a plurality of corresponding operations based on the usage history from among a plurality of operation commands for controlling the operation of the target character in the object space. A command is selected (extracted), and the selected operation command is presented as an operation command candidate selected by the operator's intention while referring to the registered spectator information.

For example, the operation command candidate presentation processing unit 214 may select an operation command when selecting the operation command.
(1) Operation commands related to the immediately preceding operation command,
(2) Operation commands that have not yet been used for the screening of the content (that is, during the execution of a series of performances).
(3) Frequently used operation commands,
(4) Operation commands and excitement during the screening
(5) Operation commands for executing special effects for some spectators and individual spectators who have not been executed for other spectators.
Is selected as an operation command candidate from a plurality of operation commands, and the selected operation command candidate is displayed on the monitor display unit 290 and presented to the operator.

Further, the operation command candidate presentation processing unit 214 also displays the spectator information on the monitor display unit 290 and presents it to the operator as described above, but the spectator information required depending on the situation such as in the case of (5). Is changed and presented to the operator.

The operation command candidate presentation processing unit 214 may simply present the operation commands corresponding to (1) to (4), or is set in advance when the operation commands are prioritized. It may be added up with the priority based on other factors such as the value decreasing depending on the priority and the number of selections, and the presentation order of the selected operation command candidates may be determined based on the priority.

Further, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, for example, the used operation commands cannot be sequentially presented, or the priority is lowered so that the operation commands can be presented or have a high priority. The operation command may be displayed on the monitor display unit 290 as an operation command candidate and presented to the operator. However, the operation command candidate presentation processing unit 214 excludes the same operation command in the same spectator for the operation command for some spectators or individual spectators.

Further, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, the operation command candidate presentation processing unit 214 displays information such as the number of times of use and the usage timing on the monitor display unit 290 together with the corresponding operation command and presents the information to the operator. May be good.

On the other hand, the operation command candidate presentation processing unit 214 performs the second operation command candidate presentation processing based on the spectator information and the usage history detected from among a plurality of operation commands for controlling the operation of the target character in the object space. A plurality of applicable operation commands are extracted, and the extracted operation commands are presented as operation command candidates.

Further, the operation command candidate presentation processing unit 214 may use a plurality of operation commands (for example, the operation commands (1) to (4) described above) as the operation command candidate presentation process, similarly to the first operation command candidate presentation process. Is selected, an operation command is further selected from the selected operation commands according to the audience information, and the selected operation command is displayed on the monitor display unit 290 as an operation command candidate and presented to the operator.

Similar to the first operation command candidate presentation process, the operation command candidate presentation processing unit 214 displays information such as the number of times of use and the timing of use together with the corresponding operation command when the operator selects the operation command. It may be displayed on 290 and presented to the operator.

(Operation command candidate presentation processing based on the usage history of operation commands for each content)
When the content screening execution control unit 218 executes the screening of the content, the content screening execution control unit 218 acquires the usage history information of the operation command already registered in the server device 11 or the like in the same content (that is, the same content ID), and the content. It is stored in the data storage unit 274.

Then, when presenting the operation command candidate, the content screening execution control unit 218 can select the operation command according to the spectator information and the registered usage history information by the operator, and the spectator information and the usage history information. The operation command candidate presentation process for presenting at least one of the operation commands selected according to is executed as an operation command candidate to the operator.

In particular, the operation command candidate presentation processing unit 214 performs, as the first operation command candidate presentation processing, a plurality of corresponding operations based on the usage history from among a plurality of operation commands for controlling the operation of the target character in the object space. The selected operation command is presented as an operation command candidate as an operation command that can be selected by the operator's intention while referring to the spectator information.

For example, the operation command candidate presentation processing unit 214 may select an operation command when selecting the operation command.
(1) Operation commands that are frequently used at the same timing in the algorithm,
(2) Operation commands that were excited at the same timing in the algorithm,
(3) Operation commands that regulate the behavior of the target character that most of the spectators have not yet seen.
Is selected as an operation command candidate from a plurality of operation commands, and the selected operation command candidate is displayed on the monitor display unit 290 and presented to the operator.

The operation command candidate presentation processing unit 214 may simply present the operation commands corresponding to (1) to (3), give priority to the operation commands, and give a preset priority. The presentation order of the operation command candidates selected based on the priority may be determined in combination with the priority based on other factors such as the value decreasing depending on the degree and the number of selections.

Further, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, for example, the used operation commands cannot be sequentially presented, or the priority is lowered, and the operation that can be presented or has a high priority is performed. The command may be displayed on the monitor display unit 290 as a selected operation command candidate and presented to the operator. However, the operation command candidate presentation processing unit 214 excludes the same operation command in the same spectator for the operation command for some spectators or individual spectators.

Further, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, the operation command candidate presentation processing unit 214 displays information such as the number of times of use and the usage timing during the screening on the monitor display unit 290 together with the corresponding operation command to the operator. You may present it.

On the other hand, the operation command candidate presentation processing unit 214 corresponds to the second operation command candidate presentation processing based on the spectator information and the usage history from among a plurality of operation commands for controlling the operation of the target character in the object space. Select multiple operation commands and present the selected operation commands as operation command candidates.

Further, the operation command candidate presentation processing unit 214 may use a plurality of operation commands (for example, the operation commands (1) to (3) above) as the operation command candidate presentation process, similarly to the first operation command candidate presentation process. Is selected, an operation command is further selected from the selected operation commands according to the audience information, and the selected operation command is displayed on the monitor display unit 290 as an operation command candidate and presented to the operator.

Similar to the first operation command candidate presentation process, the operation command candidate presentation processing unit 214 includes information such as the number of times of use and the timing of use together with the corresponding operation command when the operator is made to select the operation command being screened. It may be displayed on the monitor display unit 290 and presented to the operator.

[4.5] Other Functions and Processes Next, an operation command candidate presentation process based on the usage history of the operation commands of the present embodiment will be described.

(Behavioral control of acting characters for a specific audience)
The touch detection processing unit 211 interlocks with the viewing position information indicating the viewing position of each spectator displayed on the monitor display unit 290, for example, and specifies the viewing position of a specific spectator who enjoys the movement of the acting character together with the operation command. Audience designation instructions may be accepted.

Then, in this case, the content screening execution control unit 218 controls the operation of the acting character based on the received operation command toward the viewing position of the spectator specified by the received spectator designation instruction.

For example, the content screening execution control unit 218 approaches (approaches) the nearest position on the stage, such as hand gesture control by the acting character, line-of-sight control (control to send the line of sight), toward the specified viewing position. (Displayed in) and controls each action of the call with a given line (eg, the name of the spectator if the spectator information reveals the name of the spectator).

Further, the monitor display control unit 213 causes the monitor display unit 290 to display the position (for example, a seat) of each spectator in the screening venue as viewing position information by touch operation. Then, when the position of the spectator displayed by the touch detection processing unit 211 is touched and accepted as the spectator designation instruction, the content screening execution control unit 218 issues an operation command received toward the accepted position of the spectator. Controls the movement of the acting character based on it.

The spectator information detection unit 250 may detect the spectator information at this time and register it in the content data storage unit 274 or the server device 11 as usage history information.

Further, in the present embodiment, the reception of the viewing position of the spectator is executed by the touch operation of the touch detection processing unit 211, but instead of the touch operation, the content screening execution control unit 218 uses a trackball or a keyboard. The viewing position may be accepted by moving the cursor such as.

(Special operation command)
The operation command candidate presentation processing unit 214 may include an operation command (that is, a special operation command) determined according to the operation status of the acting character in the operation command candidate and present it to the operator.

For example, the operation command candidate presentation processing unit 214
(1) At the start of the screening of the content (that is, at the start of the operation by the algorithm), an operation command (an operation command for executing a fixed pose) for controlling the action for enlivening the audience is presented.
(2) At the end of the screening of the content, an operation command (operation command for executing a continuous call) for maintaining it and controlling the action for further excitement is presented to the excited audience.
(3) For a quiet scene such as between songs, an operation command specialized for the scene such as "line" may be presented as an operation command candidate.

The operation command candidate presentation processing unit 214 may present only the special operation command as an operation command candidate.

On the other hand, the operation command candidate presentation processing unit 214 is being screened obtained by analyzing the content screening result information collected from each content screening system 10 for each content by the server device 11 (specifically, the content management unit 102). The optimum operation command at a given timing in the content of the above may be acquired, and the acquired optimum operation command may be presented as a special operation command by being included in the operation command candidates.

(Presentation of operation command candidates based on elapsed time or number of receptions)
The operation command candidate presentation processing unit 214 changes the type of the operation command presented as the operation command candidate according to the elapsed time from the start of the content screening, the number of times the operation command is received, or the combination of the operation commands already selected. You may let me.

That is, the operation command candidate presentation processing unit 214
(1) When the screening end time approaches, an operation command that defines the operation toward the ending is displayed on the monitor display unit 290 as an operation command candidate and presented to the operator.
(2) When the number of times the operation command is accepted increases, the operation command having a short operation time is displayed on the monitor display unit 290 and presented to the operator in order to adjust the screening time of the content.
(3) According to the combination of the already selected operation commands, the operation commands whose contents match may be displayed on the monitor display unit 290 and presented to the operator.

In addition, each operation command is executed with acceptance allowance information indicating the time and interval at which acceptance is permitted when the operation control of the acting character is executed according to the algorithm, and the operation control of the acting character based on each operation command. The execution time may be specified, and the presentation of operation command candidates may be executed using this information.

Specifically, for each operation command, as the acceptance allowance information, the reception indicating the time (elapsed time) from which the performance start timing (image generation start timing) of the acting character is allowed to accept the operation command according to the algorithm. When the permissible completion time or a plurality of given sections (prelude in the case of music, verse, chorus, act 1 or 2 in the case of drama, etc.) are set and the image generation is executed. Section information indicating the section in which the operation command is allowed to be accepted is specified in.

Then, the operation command candidate presentation processing unit 214 presents the operation command according to the elapsed time from the start timing of image generation in the acting character or the section to which the timing of presenting the operation command belongs and the execution time of each operation command. An operation command in which the type of the command is determined or the content of the motion control of the acting character is changed may be presented.

That is, the operation command candidate presentation processing unit 214 is the elapsed time from the start timing, or, for example, when the operation control is a live performance in a musical piece, each part (section) such as a prelude, an A melody, a B melody, or a chorus. It is possible to identify which part of the image is the image and present the operation command, and depending on the situation, it is possible to change the content of the operation control of the target character in the operation command such as shortening the time and simplifying the operation content. It has become.

(Batch selection processing of multiple operation commands)
The content screening execution control unit 218 receives a plurality of operation commands by the touch detection processing unit 211 during the same timing or a predetermined period (for example, 10 seconds or 20 seconds) assumed to be the same timing. If so, the plurality of accepted operation commands may control the operation of the target character to be displayed according to a given order (for example, input order).

That is, for example, when it is necessary to input a plurality of operation commands in the interlocking operation in the performance performance, the operation command candidate presentation processing unit 214 issues each operation command at each timing when the continuous operation is started during the screening. When presenting, selecting, and accepting, not only the operation of the operator becomes complicated, but also the operation command candidate presenting process itself is burdened, and it may not be possible to perform accurate processing.

Therefore, the operation command candidate presentation processing unit 214 may execute the operation command candidate presentation processing at each timing and accept a plurality of operation commands in order to select a plurality of operation commands in advance.

The operation command candidate presentation processing unit 214 may accept the operation command at each timing by the operation command candidate presentation processing before the content is shown.

(Edit operation command)
The content editing processing unit 217 has a configuration for editing the operation control of the target character corresponding to each operation command based on the instruction of the operator.

Specifically, the content editing processing unit 217 adds, partially deletes, or changes the speed of performing the action of the acting character with respect to the operation command designated as the editing target based on the instruction of the operator. , Executes the integration of the operation contents of other operation commands.

For example, when the action of the acting character in the operation command A is only a fixed pose, the content editing processing unit 217 adds a motion of taking a predetermined step after the fixed pose and an action of issuing a predetermined line, and the determined pose is added. To generate an operation command A that defines the actions of a series of acting characters that perform, take steps, and emit dialogue.

Then, when the content screening execution control unit 218 receives the operation command A, the acting character controls the operation of a series of acting characters that make a determined pose, take a step, and emit a line.

Further, the content editing processing unit 217 may adjust the operation speed while maintaining the operation control of the acting character in the operation command A. For example, the content editing processing unit 217 may edit the speed so that a series of moving fixed poses executed in 10 seconds is executed in 20 seconds.

(Operation start display of operation command)
The operation command candidate presentation processing unit 214 receives an operation command at the timing during execution of the operation command candidate presentation process according to the algorithm in order to prevent the operator from forgetting to input and improve the operability of the operator. The operation start time at which the operation control of the acting character is started may be presented based on.

(Random presentation of operation commands)
In the operation command candidate presentation processing unit 214, the operation command candidate presentation processing unit 214 selects from a plurality of operation commands presented by the first operation command presentation processing or the second operation command presentation processing in order to prevent unexpectedness and patterning. Some operation commands may be randomly presented as operation command candidates.

(Control of multiple acting characters)
When the content screening execution control unit 218 controls the operation of a plurality of acting characters and performs the operation control based on the operation command, the content screening execution control unit 218 receives the operation command for one acting character and is based on the operation command. The motion control process for controlling the motion is executed, and the motion control specified in the algorithm is executed for the other acting characters.

Further, in this case, the operation command candidate presentation processing unit 214 executes the presentation for the acting character operator who executes the operation control processing based on the operation command to select, and the content screening execution control unit 218 is selected. Executes motion control processing based on operation commands for acting characters.

Further, the operation command candidate presentation processing unit 214 operates for a plurality of operators (for example, two operators) when controlling the operation of a plurality of acting characters (for example, two acting characters). The command candidate presentation process may be executed.

In this case, the operation command candidate presentation processing unit 214 presents an operation command whose control period of the operation control process based on the operation command matches as an operation command candidate so that the operation control of each acting character does not break down, and determines the operation command candidate. The process for accepting each operation command is executed by the timing of.

Then, in this case, the content screening execution control unit 218 executes the operation control of each acting character from the same predetermined timing.

(Acceptance processing of other operation commands during operation control based on operation commands)
The operation command candidate presentation processing unit 214 receives another operation command during the operation control of the acting character based on the received first operation command, and the received operation command (hereinafter referred to as "second operation command"). Operation command candidates may be presented so that operation control based on.) Can be executed.

For example, when the movement control of the acting character is executed based on the first operation command, and when the spectator information indicating that the spectator is not excited is presented, or when the first operation command is performed. The operation command candidate presentation processing unit 214 is selected according to the spectator information in a special situation such as a case where the entire performance performance time is delayed when the operation control of the performance character by the operation command is executed to the end. At least one or more operation commands are presented as operation command candidates.

Then, the content screening execution control unit 218 receives the first operation command by the touch detection processing unit 211 during the operation control of the acting character based on the first operation command (hereinafter, referred to as “first operation control”). If a different second operation command is accepted, it is canceled when the first operation control is accepted, and the action control of the acting character based on the second operation command (hereinafter, "second operation control"). ) Is activated.

When the content screening execution control unit 218 cancels the first motion control, the content screening execution control unit 218 executes the control until the motion of the acting character is well separated, and then executes the control to stop the motion.

Further, the operation command candidate presentation processing unit 214 has an operation command (that is, an operation command that has not been accepted) different from the first operation command when the first operation command is presented as the second operation command candidate. ) May be presented (the state in which the first operation command is presented may be maintained), or a predetermined operation such as according to the audience information or the usage history information of the operation command. A new command may be presented.

[4.6] Modification Example (Modification Example 1)
In the above embodiment, the operation command candidate presentation processing unit 214 displays the operation command candidate on the monitor display unit 290 so that it can be touch-operated, but it may be presented by assigning the operation command to a physical switch or keyboard. good.

In this case, the operation command candidate presentation processing unit 214 may present the switch or each key assigned to the monitor display unit 290, or may directly present the switch or each key.

(Modification 2)
In the above embodiment, the touch detection processing unit 211 specifies the viewing position based on the touch operation on the monitor display unit 290, but the monitor display unit 290 is specified by an operator such as a joystick or a keyboard. The viewing position may be specified by operating the cursor displayed on or the indicated position display.

(Modification example 3)
In the above embodiment, the content is screened to the audience who came to the event venue, but in the above embodiment, the video of the content is distributed by the network or by the television broadcasting. It can also be applied in distribution.

In this case, the content screening execution control unit 218 controls the operation of the acting character based on the algorithm or the operation command, while the communication control unit (not shown) uses the virtual camera in the object space including the acting character as a viewpoint. A data stream is generated from the image generated by the drawing unit 220, and the generated data stream is distributed to the server device 11 or a terminal device (not shown) via the communication unit 296 via a network or by a broadcast wave.

(Modification example 4)
In the above embodiment, as a production object (one aspect of the character) instead of the acting character, for example, a special effect device such as a bazooka type or a cannon cannon type cracker that shoots decorations such as confetti or tape, or a pin spot type. Lighting devices such as lights and search lights, stage devices such as movable Hanamichi and seri, acoustic devices such as speakers, or flying devices in which directing objects fly over the audience seats, and the like may be included.

That is, the content screening system 10
(1) Control the operation of the production object placed in the real space to be viewed by the audience.
(2) From among a plurality of operation commands for controlling the operation of the effect object, an operation command selected by the operator according to the spectator information indicating the information about the spectator is presented to the operator as an operation command candidate. ,
(3) From the presented operation command candidates, accept the operation command instructed by the operator and accept the operation command.
(4) Control the operation of the effect object based on the received operation command.
It may have a configuration.

Further, in the above embodiment, the operation command that defines the motion control of the acting character is presented, but as one form of the target character, the motion control of the virtual camera when imaging the object space including the acting character is performed. You may present the specified operation command.

For example, the operation command candidate presentation processing unit 214 selects an operation command selected by the operator according to the spectator information indicating the information about the spectator from among a plurality of operation commands for controlling the operation of the virtual camera. As a candidate, the content screening execution control unit 218 may be presented to the operator and control the operation of the virtual camera based on the received operation command.

(Modification 5)
In the above embodiment, the content screening system 10 executes various processes in conjunction with the server device 11, but registers and analyzes past audience information, usage history information such as operation commands, and content screening. Each process executed by the server device 11 such as acquisition of result information may be executed, and the content may be screened as a stand-alone device.

(Modification 6)
In the above embodiment, the content is displayed on the large display device 12 so that the spectators can view the content. However, the HMD (wearable image display device) allows each spectator to visually recognize the image in the virtual three-dimensional space. The VR experience (virtual reality experience) acting character may be viewed in 3D so as to be recognized as if it exists around each spectator in the virtual space.

In this case, for example, a terminal device having an HMD unit and a control unit is provided for each spectator, and each terminal device provides an image and a sound to the HMD based on various data output from the control unit. ..

In addition, each terminal device has a configuration capable of detecting the sound, line of sight, or posture (head posture) of the wearing spectator as spectator information.

Each terminal device may be provided to a plurality of spectators by the same event venue to simultaneously screen the content, or may perform the screening of the content at different timings depending on the different space.

[5] Operation in the present embodiment Next, the operation of the operation control process based on the operation command including the operation command candidate presentation process executed by the system control device 20 of the present embodiment will be described with reference to FIGS. 10 and 11. .. 10 and 11 are flowcharts showing the operation of the operation control process based on the operation command including the operation command candidate presentation process executed by the system control device 20 of the present embodiment.

(In the case of the first operation command candidate presentation processing)
In this operation, it is assumed that the operation of the acting character arranged in the object space is executed by the algorithm of the predetermined content, and the viewing information of each spectator is acquired from the server device 11 and already stored in the content data storage unit 274. It is assumed that it has been done.

Further, in this operation, the virtual camera set in the object space is controlled according to the algorithm, and images of the acting characters in the object space with the virtual camera as the viewpoint are sequentially generated and displayed on the large display device 12. It shall be.

First, when the content screening execution control unit 218 detects that it is the reception timing of the operation command based on the algorithm (step S101), the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213 and stores the content data. The spectator information of the spectators who are visiting the event venue stored in the unit 274 is acquired and displayed on the monitor display unit 290 (step S102).

At this time, the content screening execution control unit 218 causes the timer unit 240 to start measuring the remaining time (input time limit) until the operation of the acting character starts based on the operation command.

The content screening execution control unit 218 may acquire the spectator information of the spectators who are visiting the event venue directly from the server device 11.

Next, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213, and selects an operation command from a plurality of operation commands predetermined by the algorithm according to the spectator information displayed by the operator. , As an operation command candidate, it is displayed on the monitor display unit 290 and presented to the operator (step S103).

At this time, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213, and the remaining time (input limit) until the operation start time when the operation control of the acting character is started based on the operation command received at the timing. Time) is displayed on the monitor display unit 290 together with the operation command candidates and the audience information.

Next, the content screening execution control unit 218 determines whether or not the input time limit has reached "0" (step S104), and if it determines that the input time limit has reached "0", it becomes a target. The acting character is started to execute the motion control defined in advance by the algorithm (step S105), and this motion is terminated.

On the other hand, when the content screening execution control unit 218 determines that the input time limit has not yet reached "0", the touch detection processing unit 211 determines whether or not the operation command instructed by the operator is detected. If the determination is made (step S106) and it is determined that the operation command is not detected, the process returns to the process of step S104.

Further, when the touch detection processing unit 211 determines that the operation command instructed by the operator has been detected, the touch detection processing unit 211 accepts the detected operation command (step S107).

Next, the content screening execution control unit 218 starts the operation control of the target acting character based on the received operation command (step S108), and ends this operation.

(In the case of the second operation command candidate presentation process)
In this operation, it is assumed that the operation of the acting character arranged in the object space is executed by the algorithm of the predetermined content, and the viewing information of each spectator is acquired from the server device 11 and already stored in the content data storage unit 274. It is assumed that it has been done.

Further, in this operation, the virtual camera set in the object space is controlled according to the algorithm, and images of the acting characters in the object space with the virtual camera as the viewpoint are sequentially generated and displayed on the large display device 12. It shall be.

First, when the content screening execution control unit 218 detects that it is the reception timing of the operation command based on the algorithm (step S201), the spectator information detection unit 250 detects the spectator information of the spectators who are visiting the event venue. (Step S202).

At this time, even if the content screening execution control unit 218 controls the acting character to execute predetermined motion control such as output of a predetermined dialogue to the spectator and detects the spectator information of the spectator based on the motion control. good.

At this time, the content screening execution control unit 218 causes the timer unit 240 to start measuring the remaining time (input time limit) until the operation of the acting character starts based on the operation command.

Next, the operation command candidate presentation processing unit 214 analyzes the current state of the spectator based on the detected spectator information (step S203).

Next, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213, and from among a plurality of operation commands predetermined by the algorithm, the operation command is set as the operation command candidate according to the detected spectator information. It is displayed on the monitor display unit 290 and presented to the operator (step S204).

At this time, the operation command candidate presentation processing unit 214 interlocks with the monitor display control unit 213, and the remaining time (input limit) until the operation start time when the operation control of the acting character is started based on the operation command received at the timing. Time) is displayed on the monitor display unit 290 together with the operation command candidates and the audience information.

Next, the content screening execution control unit 218 determines whether or not the input time limit has reached "0" (step S205), and if it determines that the input time limit has reached "0", it becomes a target. The acting character is started to execute the motion control defined in advance by the algorithm (step S206), and this motion is terminated.

On the other hand, when the content screening execution control unit 218 determines that the input time limit has not yet reached "0", the touch detection processing unit 211 determines whether or not the operation command instructed by the operator is detected. If the determination is made (step S207) and it is determined that the operation command is not detected, the process returns to the process of step S205.

Further, when the touch detection processing unit 211 determines that the operation command instructed by the operator has been detected, the touch detection processing unit 211 accepts the detected operation command (step S208).

Next, the content screening execution control unit 218 starts the operation control of the target acting character based on the received operation command (step S209), and ends this operation.

[6] Others The present invention is not limited to the one described in the above embodiment, and various modifications can be made. For example, a term cited as a broad or synonymous term in a description in a specification or drawing can be replaced with a broad or synonymous term in another description in the specification or drawing.

In the present embodiment, various processes are executed in one content screening system 10, but the above-mentioned content screening system 10 may be configured by interlocking a plurality of devices.

Further, in the present embodiment, processing related to various contents is executed by the system control device 20 of the content screening system 10, but each function and the content program of the server processing unit 100 of the server device 11 are executed above. It may be realized. Further, each process may be executed independently of the content screening system 10 without interlocking with the server device 11.

The present invention includes a configuration substantially the same as the configuration described in the embodiment (for example, a configuration having the same function, method and result, or a configuration having the same purpose and effect). The present invention also includes a configuration in which a non-essential part of the configuration described in the embodiment is replaced. The present invention also includes a configuration that exhibits the same effects as the configuration described in the embodiment or a configuration that can achieve the same object. Further, the present invention includes a configuration in which a known technique is added to the configuration described in the embodiment.

As described above, the embodiments of the present invention have been described in detail, but those skilled in the art will easily understand that many modifications that do not substantially deviate from the novel matters and effects of the present invention are possible. .. Therefore, all such modifications are included in the scope of the present invention.

10: Content screening system 11: Server device 12: Large display device 20: System control device 100: Server processing unit 101: Communication control unit 102: Content management unit 103: Audience management unit 120: Input unit 130: Display unit 140: Storage Unit 142: Main storage unit 144: Content data 146: Audience information 148: Content collection data 180: Information storage medium 196: Communication unit 200: Processing unit 210: Object space setting unit 211: Touch detection processing unit 212: Content display control unit 213: Monitor display control unit 214: Operation command candidate presentation processing unit 215: Movement processing unit 216: Virtual camera control unit 217: Content editing processing unit 218: Content screening execution control unit 219: Content management unit 220: Drawing unit 230: Sound Processing unit 240: Timer unit 250: Audience information detection unit 260: Input unit 262: Detection unit 270: Storage unit 271: Main storage unit 272: Image buffer 273: Audience information 274: Content data storage unit 280: Information storage medium 290: Monitor display unit 292: Sound output unit 296: Communication unit

Claims (14)

When the target character to be viewed by the spectator is displayed by the spectator display device, the spectator display control means for controlling the display of the target character and the spectator display control means.
Audience information acquisition means for acquiring spectator information, which is information about the spectator, and
A presentation means for presenting operation command candidates for controlling the operation of the target character to a monitor display device visually recognized by the operator, and
A reception means that accepts the selection of an operation command by the operator from the operation command candidates presented by the presentation means, and a reception means that accepts the selection of the operation command by the operator.
An operation control means that controls the operation of the target character based on an operation command received by the reception means, and an operation control means.
With
The presentation means
An operation control system characterized by presenting the operation command candidate based on the spectator information acquired by the spectator information acquisition means. In the operation control system according to claim 1,
The monitor display device is further provided with a monitor display control means for displaying the target character.
The presentation means
An operation control system that presents the operation command candidate together with the target character. In the operation control system according to claim 1 or 2.
The presentation means
An operation control system that presents the audience information together with the operation command candidates. In the operation control system according to claim 3,
Further provided with a storage means for storing personal information for each spectator,
The presentation means
An operation control system that presents the spectator information according to the personal information of the plurality of spectators stored in the storage means. In the operation control system according to any one of claims 1 to 4.
Further provided with a registration means for registering and accumulating the information of the operation command received when the operation control of the target character is executed in the storage means as usage history information.
The presentation means
An operation control system that presents the operation command candidates according to the audience information and the usage history information. In the operation control system according to claim 5,
The usage history information includes an operation command and spectator information used when the operation command is received.
The presentation means
An operation control system that presents a given analysis result based on the usage history information and the operation command candidate according to the audience information. In the operation control system according to any one of claims 1 to 6,
The audience information acquisition means
The state of each of the spectators is acquired as the spectator information, and
The presentation means
An operation control system that presents the operation command candidates according to the spectator information including the state of the spectator. In the operation control system according to any one of claims 1 to 7.
The reception means
Along with the operation command, a spectator designation instruction for designating a specific spectator who enjoys the movement of the target character is received.
The motion control means
An operation control system that controls the movement of the target character toward a spectator specified by the spectator designation instruction. In the operation control system according to any one of claims 1 to 8.
The presentation means
An operation control system that includes an operation command determined according to the operation status of the target character in the operation command candidates and presents the operation command. In the operation control system according to any one of claims 1 to 9.
The presentation means
The presentation is made according to the elapsed time from the start timing of image generation in the target character, the number of times the operation command is received during the image generation, or the combination of the types of the operation command received during the image generation. An operation control system that changes the type of operation command to be performed. In the operation control system according to any one of claims 1 to 9.
When the movement of the target character is imaged while following a given algorithm,
For each operation command, the reception completion allowable time indicating the time when the reception of the operation command is permitted from the start timing of the image generation in the target character or a plurality of given sections is defined, and the image generation is executed. The section information indicating the section in which the operation command is allowed to be accepted and the execution time during which the operation control of the target character is executed based on each operation command are defined.
The presentation means
The type of the operation command to be presented is determined according to the elapsed time from the start timing of image generation in the target character, the section to which the timing for presenting the operation command belongs, and the execution time of each operation command. Alternatively, an operation control system that presents an operation command in which the content of the operation control of the target character is changed. In the operation control system according to any one of claims 1 to 11.
The motion control means
An operation control system that starts operation control of the target character based on the received operation command from a predetermined timing when an operation command is received during image generation of the target character. A spectator display control means that controls the display of the target character when the target character to be viewed by the spectator is displayed by the spectator display device.
Audience information acquisition means for acquiring spectator information, which is information about the spectator, and
A presentation means for presenting operation command candidates for controlling the operation of the target character to a monitor display device visually recognized by the operator.
A reception means that accepts the selection of an operation command by the operator from the operation command candidates presented by the presentation means, and a reception means that accepts the selection of the operation command by the operator.
An operation control means that controls the operation of the target character based on an operation command received by the reception means.
Make your computer work as
The presentation means
A program characterized by presenting the operation command candidate based on the spectator information acquired by the spectator information acquisition means. A spectator display device used to display the target character to be viewed by the spectator,
An operation control system that controls the operation of the target character,
With
The operation control system
Audience display control means for controlling the display of the target character, and
Audience information acquisition means for acquiring spectator information, which is information about the spectator, and
A presentation means for presenting operation command candidates for controlling the operation of the target character to a monitor display device visually recognized by the operator, and
A reception means that accepts the selection of an operation command by the operator from the operation command candidates presented by the presentation means, and a reception means that accepts the selection of the operation command by the operator.
An operation control means that controls the operation of the target character based on an operation command received by the reception means, and an operation control means.
With
The presentation means
A character screening system characterized by presenting the operation command candidate based on the spectator information acquired by the spectator information acquisition means.

Images