WO2019017023A1

WO2019017023A1 - Information processing device, information processing method, and program

Info

Publication number: WO2019017023A1
Application number: PCT/JP2018/015548
Authority: WO
Inventors: 文彦飯田; 龍一鈴木; 拓也池田
Original assignee: ソニー株式会社
Priority date: 2017-07-19
Filing date: 2018-04-13
Publication date: 2019-01-24

Abstract

[Problem] To provide an information processing device, an information processing method, and a program with which it is possible to recognize a spatial layout, and evaluate the compatibility of projection domains on the basis of environment information. [Solution] An information processing device comprising: a recognition unit that recognizes a spatial layout on the basis of data from an environment sensor; and an evaluation unit that evaluates the projection compatibility of each projection domain in a space on the basis of the recognized spatial layout and the data from the environment sensor.

Description

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM

The present disclosure relates to an information processing device, an information processing method, and a program.

In recent years, with the miniaturization and high brightness of projectors, the introduction for consumers and business is progressing. When introducing a projector, it is essential to secure and optimize the projection area, but conventionally, it has been necessary for the user to manually adjust the projection area.

With regard to adjustment of the projection area, for example, Patent Document 1 below discloses a projection type display device that automatically determines a projectable area. Such a projection display apparatus can detect an obstacle or the like by using an area sensor such as a depth sensor, and can determine a projectable area.

Further, Patent Document 2 below discloses a projector-projector-type autonomous robot capable of moving itself to a position where a human who interacts can easily view it and projecting it onto a searched optimum projection plane. Such an autonomous robot obtains information on whether or not the projection area is a plane, the color tone of the projection area, and the degree of gloss, and determines the optimum projection plane.

JP, 2015-145894, A JP 2005-313291 A

However, the above-mentioned patent document 1 is limited to the estimation of the propriety of the projection based on the recognition of the obstacle, and the judgment of the adaptability including the surrounding environment such as the illuminance and the material of the projection surface is not performed. Was not enough.

Moreover, in the said patent document 2, it is assumed that a projection apparatus will be mounted in a single self-propelled robot, and is not considered regarding projection apparatuses other than self-propelled type.

Thus, the present disclosure proposes an information processing apparatus, an information processing method, and a program capable of recognizing a spatial layout and evaluating the suitability of a projection area based on environmental information.

According to the present disclosure, a recognition unit that recognizes a spatial layout based on data from an environmental sensor, and evaluation of projection suitability of a projection area in space based on the recognized spatial layout and data from the environmental sensor An information processing apparatus including: an evaluation unit that

According to the present disclosure, a processor recognizes spatial layout based on data from an environmental sensor, and projection suitability of a projection area in space based on the recognized spatial layout and data from the environmental sensor Propose an information processing method including the evaluation of

According to the present disclosure, a computer is a recognition unit that recognizes a spatial layout based on data from an environmental sensor, and projection adaptation of a projection area in space based on the recognized spatial layout and data from the environmental sensor. We propose a program to make it function as an evaluation unit that evaluates gender.

As described above, according to the present disclosure, it is possible to recognize the spatial layout and evaluate the suitability of the projection area based on the environment information.

Note that the above-mentioned effects are not necessarily limited, and, along with or in place of the above-mentioned effects, any of the effects shown in the present specification, or other effects that can be grasped from the present specification May be played.

It is a figure explaining an outline of an information processing system by one embodiment of this indication. It is a block diagram showing an example of composition of an information processor by this embodiment. It is a figure explaining the conditions at the time of evaluating the conformity of the projection field by this embodiment. It is a figure which shows an example of the projection compatibility distribution by this embodiment. It is a figure which shows an example of the projection suitability evaluation factor by this embodiment. It is a flowchart which shows the operation processing of the projection system by this embodiment. It is a figure which shows the whole structure of the projection system by the modification of this embodiment. It is a figure explaining the case where projection area whole surface can not be projected. It is a figure explaining the cooperation operation | movement by the several projection display apparatus by this embodiment. It is a figure explaining other cooperation operation by a plurality of projection display devices by this embodiment. It is a figure explaining the illumination intensity nonuniformity of a projection field. It is a figure explaining the case where unevenness of illumination intensity (environment illumination intensity) or color tone exists in a projection field. It is a figure explaining other cooperation operation by a plurality of projection display devices by this embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration will be assigned the same reference numerals and redundant description will be omitted.

The description will be made in the following order.
1. Overview of an information processing system according to an embodiment of the present disclosure Configuration of Information Processing Device 10 2-1. Description of Configuration 2-2. Evaluation of projection suitability 3. Operation processing 4. Modified Example 4-1. Configuration 4-2. Cooperative operation example 5. Summary

<< 1. Overview of Information Processing System According to One Embodiment of the Present Disclosure >>
FIG. 1 is a diagram for describing an overview of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 1, a projection system 1 according to the present embodiment includes an information processing device 10, a projection display device 11, and an environment measurement device 12. Hereinafter, each device included in the projection system 1 according to the present embodiment will be described.

(Projection display device 11)
The projection display device 11 has a function of projecting display information in accordance with control from the information processing device 10. The projection display apparatus 11 is realized by, for example, a so-called projector, and projects display information (for example, image information) to be presented on a projection area (real object). The display information is transmitted from, for example, the information processing apparatus 10. The projection display device 11 may be realized by a head-up display or a retinal projection display.

In addition, the projection display apparatus 11 may further have a zoom mechanism, an autofocus function, and the like. Moreover, the projection display apparatus 11 may be single or plural. Also, the projection display device 11 may be fixed or movable. In addition, the projection display device 11 may be realized by a so-called moving projector which can drive the projection direction vertically or horizontally and project the display information in any direction.

(Environmental measuring device 12)
The environment measuring device 12 detects surrounding environment information and transmits it to the information processing device 10 and the projection display device 11. Specifically, the environment measurement device 12 has an electronic camera and a distance sensor, and acquires environment information of the projection area. The environment measurement device 12 may be single or plural. In addition, the environment measurement device 12 may be provided in the projection display device 11.

The electronic camera captures an image of a projection area to obtain a captured image. The electronic camera may be single or plural. The imaging wavelength is not limited to the visible light range, and may include ultraviolet light and infrared light, or may be limited to a specific wavelength range. Also, it may be a sensor that measures only the illuminance.

The distance sensor acquires a projection distance (a distance from the projection display device 11 to the projection area). As a distance sensor, a method using infrared light as a light source, a method using ultrasonic waves, a method using a plurality of cameras, a method using image processing, and the like can be mentioned. The present embodiment will be described using an infrared distance sensor as an example. Also, the distance sensor may be single or plural, or may collectively acquire distance information in the area.

The electronic camera and the distance sensor constituting the environment measurement device 12 may be provided at different places or at the same place. For example, an electronic camera and a distance sensor may be provided in the projection display device 11, or only the distance sensor may be provided in the projection display device 11 and the electronic camera may be provided at another place in the space.

(Information processing apparatus 10)
The information processing device 10 transmits display information to the projection display device 11, and controls the projection by the projection display device 11 so as to project the display information on a predetermined projection area. In addition, the information processing apparatus 10 can also perform correction on display information as appropriate. The information processing apparatus 10 may be single or plural.

Further, the information processing apparatus 10 according to the present embodiment acquires the material of the projection area and the illuminance environment from the various environment information (specifically, the captured image and the distance information) measured by the environment measurement apparatus 12, and Assess the projection suitability of The information processing apparatus 10 determines an optimal projection area based on the evaluation result of the projection compatibility, and controls the projection display apparatus 11 to project display information on the projection area, thereby ensuring visibility. Environment can be realized. Further, the information processing apparatus 10 can improve the visibility by appropriately correcting the display information in accordance with the evaluation of the projection compatibility of the projection area.

The specific configuration of the information processing apparatus 10 according to the present embodiment will be described later with reference to FIG.

The configuration of the projection system 1 according to the present embodiment has been described above. The devices of the projection system 1 may be installed at different places or at the same place.

Moreover, each device of the projection system 1 may be separately configured, or at least some of the devices may be configured as the same device. For example, it may be a projection apparatus including all of the information processing apparatus 10, the projection display apparatus 11, and the environment measurement apparatus 12, or the projection display apparatus 11 and the environment measurement apparatus 12 may communicate with the information processing apparatus 10 as one unit. You may do so.

Further, the information processing apparatus 10 and the projection display apparatus 11 may be connected in communication with the environment measurement apparatus 12 as one body.

<< 2. Configuration of Information Processing Device 10 >>
Subsequently, a specific configuration of the information processing apparatus 10 included in the projection system 1 of the present embodiment will be described with reference to FIG.

<2-1. Configuration Description>
FIG. 2 is a block diagram showing an example of the configuration of the information processing apparatus 10 according to the present embodiment. As shown in FIG. 2, the information processing apparatus 10 includes a control unit 100, a communication unit 110, an input unit 120, an output unit 130, and a storage unit 140.

The control unit 100 functions as an arithmetic processing unit and a control unit, and controls overall operations in the information processing apparatus 10 according to various programs. The control unit 100 is realized by, for example, an electronic circuit such as a central processing unit (CPU) or a microprocessor. The control unit 100 may also include a ROM (Read Only Memory) that stores programs to be used, operation parameters, and the like, and a RAM (Random Access Memory) that temporarily stores parameters and the like that change appropriately.

Further, the control unit 100 according to the present embodiment also functions as a space recognition unit 101, a projection suitability evaluation unit 102, a projection area determination unit 103, a correction unit 104, and a projection control unit 105.

The space recognition unit 101 acquires object information in the surrounding space based on the environment information acquired by the environment measurement device 12 (object recognition), and recognizes the layout of the surrounding space. The space recognition unit 101 according to the present embodiment can perform space recognition in real time. For example, the space recognition unit 101 can recognize the current spatial layout based on the current captured image captured by the electronic camera and the current distance information acquired by the distance sensor. Such spatial layout recognition may be performed continuously or may be performed at the start of projection. A generally known algorithm may be used as an algorithm for object recognition, and is not particularly limited in the present embodiment.

The projection suitability evaluation unit 102 evaluates the projection suitability in the projection area based on the environment information acquired by the environment measurement device 12. The projection area indicates a real object which the projection display device 11 actually projects and displays display information. The projection suitability evaluation unit 102 may evaluate projection suitability for each real object recognized by the space recognition unit 101 (for example, a wall, a window, a television, a table, a chair, a floor, a ceiling, etc.).

The projection suitability evaluation unit 102 more reliably evaluates the suitability of the projection area based on the material of the projection area acquired by analyzing the captured image which is the environmental information acquired by the environment measuring device 12 and the illuminance environment and the like. Do. In addition, the projection suitability evaluation unit 102 uses the projection distance information which is environment information acquired by the environment measurement device 12, the luminance information acquired from the environment measurement device 12, the color information of the projection image, etc. You may do sex evaluation. A specific example of the compatibility evaluation according to the present embodiment will be described later with reference to FIG. The projection suitability evaluation unit 102 may dynamically (in real time) reflect the feedback from the environment measurement device 12 (acquisition of a captured image obtained by capturing a projection image) in the evaluation of projection suitability.

The projection area determination unit 103 determines the projection area on which the display information is projected, based on the result of the compatibility evaluation by the projection suitability evaluation unit 102. For example, the projection area determination unit 103 may determine the area (real object) where the value of the suitability evaluation is the highest, or may determine the area where the suitability evaluation result satisfies the predetermined condition. .

The correction unit 104 corrects the display information according to the determined projection area and the suitability of the projection area. For example, the correction unit 104 can correct the shape of the display information (for example, distortion correction) according to the projection area, or adjust the luminance of the display information according to the adaptability of the projection area.

The projection control unit 105 transmits the display information to the projection display device 11, and performs control to project the display information on the determined projection area. Specifically, for example, the projection control unit 105 transmits a signal for driving the projection direction to project the display information on the determined projection area based on the recognized spatial layout and the position (known) of the projection display device 11 Do. Alternatively, the projection control unit 105 may transmit the information on the spatial layout and the information on the position of the determined projection area to the projection display device 11 so that the drive control in the projection direction is performed on the projection display device 11 side.

(Communication unit 110)
The communication unit 110 transmits and receives data to and from an external device directly or through a network by wire or wirelessly. For example, the communication unit 110 communicates with the projection display device 11 or the environment measuring device 12 by using a wired / wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), short distance wireless communication, or the like. Do.

(Input unit 120)
The input unit 120 receives input information input by the user to the information processing apparatus 10. For example, the input unit 120 may be an operation input unit that receives an operation instruction from a user. The operation input unit may be a touch sensor, a pressure sensor, or a proximity sensor. Alternatively, the operation input unit may be a physical configuration such as a button, a switch, and a lever. Also, the input unit 120 may be a voice input unit (microphone).

(Output unit 130)
The output unit 130 outputs presentation information to the user. For example, the output unit 130 may be realized by a display unit or an audio output unit (microphone). The display unit outputs an operation screen, a menu screen or the like, and may be a display device such as a liquid crystal display (LCD) or an organic electro luminescence (EL) display.

(Storage unit 140)
The storage unit 140 is realized by a ROM (Read Only Memory) that stores a program used for processing of the control unit 100, calculation parameters, and the like, and a RAM (Random Access Memory) that temporarily stores parameters and the like that change appropriately.

The storage unit 140 stores display information (content). The display information may be transmitted from an external device via the communication unit 110.

The configuration of the information processing apparatus 10 according to the present embodiment has been specifically described above. The configuration of the information processing apparatus 10 according to the present embodiment is not limited to the example shown in FIG. For example, the information processing device 10 may be configured by a plurality of devices. Also, analysis of environmental information may be performed on the environmental measurement device 12 side. In addition, the information processing apparatus 10 may be realized by a client apparatus or may be realized by a server on a network.

<2-2. Evaluation of projection suitability>
Next, the evaluation of the projection suitability described above will be specifically described with reference to FIG. The projection suitability evaluation unit 102 according to the present embodiment more reliably evaluates the suitability of the projection area based on various factors obtained by analyzing the environmental information.

FIG. 3 is a diagram for explaining conditions (factors) when evaluating the suitability of the projection area.

(Condition 1: If surveying is impossible)
Condition 1 in FIG. 3 shows the case where distance measurement with the distance sensor is not possible. In this case, it is assumed that the state of the projection area is "the distance between the projection area and the distance sensor is out of the measurement range of the distance sensor" or "the projection area is a material which can not be measured by the distance sensor".

For example, in the case where “the distance between the projection area and the distance sensor is out of the measurement range of the distance sensor” by matching or correlating the measurement range of the distance sensor to be used with the projection distance of the projection display 11 in advance It can be determined that the projection suitability of the projection area is low. More specifically, by matching the lower limit of the distance sensor and the lower limit of the projection distance of the projection display 11, for example, the projection compatibility can be determined by whether the distance can be measured.

Also, for example, when the distance sensor is of the infrared type, measurement of an infrared transmitting material (glass, acrylic resin, etc.), an infrared absorbing material (black body, etc.), infrared scattering, reflective material (a carpet, earth wall, mirror, etc.) can not be measured. It corresponds to the material. Since the above-mentioned non-measurable material in such an infrared method is a material not suitable for projection by the projection display device 11, it can be judged that the projection compatibility is low. Although the material which can not be measured is different depending on the principle of the distance sensor used, it is possible to assume the material which can not be measured of each sensor, and therefore, the material and the surface property of the projection area (real object) can be estimated. Also, the projection suitability may be evaluated in combination with information (luminance information, color information, object information, etc.) obtained from a separately acquired captured image. In the case of a distance sensor using a camera, the material and surface properties of the projection area may be determined based on information such as object recognition from a captured image in addition to distance information to evaluate projection suitability.

(Condition 2 and Condition 3: Illuminance environment)
When Condition 2 and Condition 3 in FIG. 3 are both capable of distance measurement and the color tone of the projection area is dark such as black from the captured image information acquired by the electronic camera, the evaluation of the illuminance environment is different Indicates

First, when the color tone of the projection area is dark, the visibility is lowered, and therefore the condition 2 and the condition 3 are evaluated as having low projection suitability from the viewpoint of "the color tone of the projection area". The determination of the light and dark color of the projection area can be made, for example, by converting captured image information obtained from the electronic camera into a gray scale or the like. In addition, when the gray scale values of the projection area are close, the projection suitability may be evaluated based on the color information and the like of the base color of the display image to be projected.

Then, when the projection area is in proximity to the projection display device 11 or when the environment illuminance (illuminance around the projection area) is sufficiently low, the visibility is improved, but usually, according to the relationship between the environment illuminance and the projection illuminance. Visibility changes. Here, the illuminance (projection illuminance) projected from the projection display device 11 to the projection area is determined by the luminance and projection distance unique to the projection display device, and is defined as the projection distance from the projection display device 11 to the projection area It decreases in inverse proportion. Therefore, depending on the installation position of the projection display device 11, the threshold value of the ambient environment illuminance that degrades the visibility differs. The projected illumination may be calculated in consideration of the color tone of the image to be projected.
(Formula 1)
Projection illuminance = f (projection display luminance, projection distance)

In the present embodiment, the illuminance environment is evaluated by comparing the projected illuminance and the environmental illuminance, and the evaluation of the illuminance environment is used for the evaluation of the projection suitability. Table 1 below shows an example of illuminance evaluation based on comparison of environmental illuminance and projected illuminance. Such illuminance evaluation is an example, and the present embodiment is not limited to this.

The projection suitability evaluation unit 102 can evaluate the projection suitability in consideration of the illuminance evaluation. The projection suitability evaluation method may be a function using projection illumination and environmental illumination.
(Formula 2)
Projection suitability = f (environmental illumination, projection illumination)

Moreover, as a method of illuminance evaluation, it is also assumed to acquire luminance information from captured image information by an electronic camera, or to perform by combining a visible light image and an infrared image.

In the example shown in FIG. 3, in condition 2 and condition 3, the illuminance evaluation is higher in condition 2 and the other conditions (surveying, color tone of the projection area) are the same. It is evaluated as high.

(Condition 4 and Condition 5: surface roughness)
Condition 4 and condition 5 in FIG. 3 indicate that distance measurement is possible, the projected area is dark such as black, and the illuminance environment is equivalent but the surface roughness of the projected area is different. .

The surface roughness of the projection area can be obtained, for example, by analyzing distance information acquired from a distance sensor or captured image information acquired from an electronic camera.

If the surface of the projection area is rough, the visibility is reduced. Therefore, under the conditions 4 and 5 shown in FIG. 3, the condition 4 in which the surface is smooth is evaluated as having a higher projection compatibility than the condition 5 in which the surface is rough. The influence of the surface roughness on the visibility may be evaluated by a binary value or a function.
(Equation 3)
Projection suitability = f (surface roughness)

(Condition condition 6 and condition 7 :)
Conditions 6 and 7 in FIG. 3 indicate that distance measurement is possible, the illuminance environment and the surface roughness are respectively equal, and the color tone of the projection area is different.

As described above, the color tone of the projection area can be obtained, for example, by converting the captured image information obtained from the electronic camera into a gray scale or the like. improves.

Therefore, under the above conditions 6 and 7 shown in FIG. 3, it is evaluated that the condition 6 of bright color is higher in projection compatibility than the condition 7 of dark color in the projection area. The influence of the surface tone on the visibility may be evaluated as a binary value or a function based on gray scale values, RGB values, and the like.

(combination)
The projection suitability evaluation unit 102 according to the present embodiment evaluates the projection suitability as a function by appropriately combining the plurality of evaluation factors (the survey, the color tone of the projection area, the surface roughness of the projection area, and the illumination evaluation) described above. May be

(Projected fitness distribution)
Moreover, the projection suitability evaluation part 102 by this embodiment can grasp | ascertain the projection suitability distribution in space based on evaluation of the projection suitability mentioned above. Thereby, the projection suitability evaluation unit 102 can determine the area suitable for projection. That is, the projection suitability evaluation unit 102 evaluates projection suitability for each real object to obtain a distribution of projection suitability in space.

Here, FIG. 4 shows an example of the projection compatibility distribution according to the present embodiment. For example, the projection suitability evaluation unit 102 performs projection suitability evaluation on the space layout 20 as shown in the upper part of FIG. 4 and generates a projection suitability distribution 21 as shown in the lower part of FIG. 4. The projection compatibility distribution 21 shown in the lower part of FIG. 4 indicates the height of compatibility according to the color density, and for example, the projection area having the highest color density among the projection area 211, the projection area 212, and the projection area 213. 213 is the best fit, followed by the projection area 212, and the projection area 211 is less fit.

Thus, the projection suitability evaluation unit 102 can determine that the projection area 213 is the most suitable part for projection. Although the projection compatibility distribution is shown by color density as an example here, the present embodiment is not limited to this, and for example, a projection compatibility distribution shown by a color distribution may be generated. Also, the generated projection suitability distribution may be presented to the user.

(Example of the factor of projection suitability evaluation)
Regarding the factors of the projection suitability evaluation, in the specific example described with reference to FIG. 3, “Gearning, color tone of projection area, surface roughness of projection area, and illuminance evaluation” are mentioned, but this embodiment is not limited to this. Not limited to, other factors are also conceivable.

Here, FIG. 5 shows an example of the projection suitability evaluation factor. The example shown in FIG. 5 also includes the factors used in the above-described specific example. The evaluation of the projection compatibility is performed based on, for example, “material of projection area”, “environment of projection area”, or “environment of projection display device”.

The material of the projection area is information on the material of the projection surface (real object), and the roughness of the projection area, the smoothness of the projection area, the reflection characteristic of the projection area, the transmission characteristic of the projection area, and the color tone of the projection area Etc.

Further, the environment of the projection area is information related to the environment of the projection surface (real object), and includes an illuminance environment.

The environment of the projection display apparatus is an environment relating to projection, and includes the distance to the projection area, the brightness information of the projection display apparatus, the color information of the projection image, and the like.

The many factors described above are acquired based on captured image information captured by an electronic camera, distance information detected by a distance sensor, device information acquired from the projection display apparatus 11, and the like.

Thus, the projection suitability evaluation unit 102 according to the present embodiment can more reliably evaluate the projection suitability based on the material of the projection area, the environment of the projection area, the environment of the projection display apparatus, and the like. .

<< 3. Motion processing >>
Subsequently, the operation processing of the projection system according to the present embodiment will be specifically described with reference to FIG. FIG. 6 is a sequence diagram showing operation processing of the projection system according to the present embodiment.

As shown in FIG. 6, first, the information processing apparatus 10 acquires environmental information from the environmental measurement apparatus 12 (step S103).

Next, the space recognition unit 101 of the information processing device 10 performs recognition of the surrounding space (specifically, object recognition) based on the environment information (step S106).

Next, the projection suitability evaluation unit 102 of the information processing device 10 evaluates projection suitability (step S109). Specifically, the projection suitability evaluation unit 102 determines the material of the projection area and the projection area based on the environment information and the device information acquired from the projection display device 11 (eg, luminance information and installation position of the projection display device 11). The color tone, the roughness of the projection area, the projection distance (the distance from the projection display device 11 to the projection area), the illuminance environment of the projection area, and the like are determined to evaluate the projection suitability.

Next, the projection area determination unit 103 of the information processing device 10 determines a projection area based on the evaluation result of the projection compatibility (step S112).

Next, the correction unit 104 of the information processing device 10 performs image correction (distortion correction and the like) of display information (as required) according to the determined projection area (step S112), and the projection display device 11 is Then, projection control is performed to project the corrected display information (step S115). Note that the information processing apparatus 10 can also instruct the projection display apparatus 11 to perform position correction (display position correction) in the projection direction so as to direct the projection direction to the determined projection area during projection control.

Heretofore, the operation processing of the projection system according to the present embodiment has been specifically described.

<< 4. Modified example >>
Subsequently, modified examples of the projection system according to the present embodiment will be described with reference to FIGS. 7 to 13. In the embodiment described above, the case of using a single projection display device 11 has been described as an example, but the present embodiment is not limited to this, and a plurality of projection display devices 11 are cooperated to further improve visibility. It is possible.

That is, even if the projection illuminance by the single projection display device 11 is not sufficient for the environment illuminance and the visibility is low, for example, the plurality of projection display devices 11 cooperate with one another to provide the same image information. By overlapping and projecting, the projection illuminance can be increased to improve the visibility.

The information processing apparatus 10 according to the present embodiment creates such a projection compatibility distribution at the time of linkage of a plurality of units, thereby making it impossible to project (for example, an area judged to be projection impossible because the projection compatibility falls below a predetermined threshold). Can be reduced and at the same time the visibility can be improved.

<4-1. Configuration>
FIG. 7 is a diagram showing an overall configuration of a projection system 1a according to the present modification. As shown in FIG. 7, the projection system 1a includes an information processing device 10a, a plurality of projection display devices 11a-1 and 11a-2, and a plurality of environment measurement devices 12a-1 and 12a-2.

Here, the configuration includes a plurality of environment measurement devices 12a respectively corresponding to a plurality of projection display devices 11a as an example, but the present modification is not limited to this, and the environment measurement device 12a may be singular. , May be three or more.

Moreover, although two projection display apparatuses 11a are shown in figure in FIG. 7, this modification is not limited to this, Three or more projection display apparatuses 11a may be sufficient.

The information processing device 10a controls the projection from the projection display device 11a according to the evaluation result of the projection compatibility. The information processing apparatus 10a can be controlled to project from a single projection display apparatus 11a, or can be controlled to perform projection by linking a plurality of projection display apparatuses 11a-1 and 11a-2. It is possible.

Similar to the operation processing described with reference to FIG. 6, the information processing apparatus 10a according to the present modification acquires environment information, evaluates projection suitability in space, and determines an optimal projection area. In the evaluation of the projection compatibility, the evaluation of the projection compatibility in the case of operating each of the projection display devices 11a-1 and 11a-2 in a single case and in the case of a plurality of cooperation (a plurality of projection images are superimposed and projected) And evaluation of the projective suitability of

The information processing apparatus 10a evaluates the projection compatibility at the time of a plurality of cooperation, for example, when the projection suitability in the case of performing projection with a single projection display device 11a is low, and based on the projection compatibility distribution at the time of a plurality of cooperation It is also possible to perform projection area determination and projection control (multiple cooperation). Hereinafter, the improvement of the visibility by a plurality of collaborations will be specifically described.

<4-2. Cooperative operation example>
(First example of linked operation)
First, with reference to FIG. 8 and FIG. 9, a first example of cooperative operation will be described.

FIG. 8 is a diagram for explaining the case where sufficient visibility can not be obtained with a single projection. As shown on the left of FIG. 8, when the entire surface of the projection area 22 can not be projected (for example, when the illuminance evaluation is low, the color tone of the projection area is dark, etc., the projection suitability is low), a single projection display device 11 a It is assumed that sufficient visibility (insufficient projection illuminance) can not be obtained as in the case of the projection image 23 shown on the right of FIG.

In this case, the information processing apparatus 10a operates the plurality of projection display apparatuses 11a-1 and 11a-2 in cooperation to project the same image at the same position (that is, all the plural same images are superimposed and projected). It is possible to obtain projected illumination and improve visibility. FIG. 9 is a diagram showing an example of projection by such cooperative operation.

First, the information processing apparatus 10a determines the projection area 25 as shown on the left of FIG. 9 according to the projection compatibility distribution (during multiple cooperation). In the example shown on the left side of FIG. 9, for example, a range that is as highly compatible as possible based on the material of the projection area, the environment, etc.

Next, the information processing device 10a controls the projection display devices 11a-1 and 11a-2 to project the same image information to the determined projection area 25. As a result, as shown on the right of FIG. 9, the projection illuminance of the projected image 26 on which a plurality of the same images are projected in an overlapping manner is increased, and the visibility can be improved.

In the case where it is difficult to superimpose the same image due to the relationship of the installation positions of the projection display devices 11a-1 and 11a-2, the information processing device 10a superimposes only a part according to the priority of the display content, It is also possible to secure gender. For example, the priority of the display content may be set to be high at a place where visibility should be particularly raised, such as text information, or may be set arbitrarily by the user.

FIG. 10 is a diagram for explaining another cooperative operation by the plurality of projection display apparatuses according to the present embodiment. As shown in FIG. 10, for example, by superimposing and projecting the text portion 28 of the projection image 27 by the plurality of projection display devices 11a, the visibility can be secured.

(Second example of linked operation)
Next, with reference to FIGS. 11 to 13, a second example of the cooperative operation will be described.

FIG. 11 is a diagram for explaining illuminance unevenness in a projection area. As shown in the left of FIG. 11, when the position of the projection display device 11 is not facing the projection area, the projection illuminance is higher as the projection distance is shorter and the projection illuminance is lower as the distance is longer. Unevenness occurs. Therefore, as shown on the right of FIG. 11, even in the case of projection by a plurality of projection displays 11a-1 and 11a-2, if the position of each projection display 11a does not face the projection area, the illuminance of the projection area Unevenness occurs. In this case, the illuminance visually recognized by each user varies, leading to the degradation of the projection quality.

Further, as shown on the left of FIG. 12, even when the projection area 30 has unevenness in illuminance (environmental illuminance) or color tone, as described above, the illuminance visually recognized by the viewer varies, and the display quality is deteriorated. Connect. Therefore, as shown on the right of FIG. 12, in order to maintain the display quality as much as possible, the projection area 32 is limited to a part of the projection area 31 (a part having a relatively high projection compatibility) and a sufficient projection area (area) Can not secure.

When uneven illuminance occurs in the projection area as shown in FIGS. 11 and 12, the information processing apparatus 10a according to the present modification adds contrast to the projected image by adding luminance correction to the projected image based on the projection compatibility distribution at the time of cooperation. It becomes possible to give priority to the projection area and improve the unevenness of illumination and to project even if.

FIG. 13 is a diagram for explaining another cooperative operation by the plurality of projection display apparatuses according to the present embodiment. As illustrated in the left of FIG. 13, the information processing apparatus 10 a determines a projection area in consideration of background luminance correction (prefers the projection area to the contrast).

Next, as shown in the center of FIG. 13, the information processing apparatus 10 a (dynamically) applies and projects background luminance correction to the projection image. Specifically, for example, when the contrast of a part (projection area 33) of the projection image (projected by the single or plural information processing apparatuses 10a) is high due to uneven illuminance in the projection area, for example, the projection area Irregularities in illuminance can be improved by projecting a projected image on which luminance correction has been performed so as to reduce the contrast of 33. Such luminance correction of the projection image can be performed by the correction unit 104 of the information processing device 10a. At this time, the information processing apparatus 10a can also perform luminance correction dynamically (in real time) based on feedback from the environment measurement apparatus 12a (acquisition of a captured image obtained by capturing a projected image).

Here, in the improvement of illuminance unevenness, even if contrast is reduced by background brightness correction, the projection area is prioritized, but what priority should be given may be defined for each display information (content) . When priority is given to contrast even if the projection area is reduced, as shown on the right of FIG. 12, it is possible to secure contrast by projecting only a partial region according to the projection compatibility distribution. .

The information processing apparatus 10a may appropriately perform projection control according to priority information (a contrast or a priority of a projection area) corresponding to each content when performing the projection control according to the projection suitability evaluation. .

<< 5. Summary >>
As described above, in the information processing system according to the embodiment of the present disclosure, it is possible to recognize the spatial layout and evaluate the suitability of the projection area based on the environment information.

More specifically, in the present system, projection suitability can be determined more reliably based on environmental information, and projection can be performed at a position highly visible to the user. For example, even when the information processing apparatus 10 according to the present embodiment performs projection by a projection display apparatus with low illuminance, an area with higher visibility (for example, a bright color tone of the projection area) according to the evaluation result of projection compatibility Visibility can be improved by projecting to environmental illumination intensity with respect to projection illumination intensity, etc.).

Further, in the information processing apparatus 10 according to the present embodiment, the projection display apparatus is controlled by lowering the output of the projection display apparatus when sufficient visibility is ensured according to the evaluation result of the projection compatibility. Energy saving is also possible.

Further, the information processing apparatus 10 can improve the projection quality by performing the correction of the (projection) position of the projection image and the image correction based on the distribution of the projection suitability. Further, the information processing apparatus 10 can acquire information of the projection distance, and can perform focus adjustment and the like.

Further, even when the projection area is divided into a plurality of parts (for example, the projection area may be divided for each real object), the information processing apparatus 10 performs the projection as the same area when the evaluation values of the projection suitability match. It is possible to do and to enlarge the projected area.

The preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, but the present technology is not limited to such examples. It is obvious that those skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims. It is understood that also of course falls within the technical scope of the present disclosure.

For example, the hardware such as the CPU, ROM, and RAM incorporated in the information processing apparatus 10, the projection display apparatus 11, or the environment measurement apparatus 12 described above includes the information processing apparatus 10, the projection display apparatus 11, or the environment measurement apparatus 12 It is also possible to create a computer program for exhibiting the functions of A computer readable storage medium storing the computer program is also provided.

In addition, the effects described in the present specification are merely illustrative or exemplary, and not limiting. That is, the technology according to the present disclosure can exhibit other effects apparent to those skilled in the art from the description of the present specification, in addition to or instead of the effects described above.

Note that the present technology can also have the following configurations.
(1)
A recognition unit that recognizes a spatial layout based on data from an environmental sensor;
An evaluation unit that evaluates projection suitability of a projection area in space based on the recognized spatial layout and data from the environment sensor;
Equipped with
Information processing device.
(2)
The environmental sensor includes one or more imaging sensors and a distance sensor.
The information processing according to (1), wherein the recognition unit recognizes the spatial layout by performing object recognition based on the image information acquired by the imaging sensor and the distance information acquired by the distance sensor. apparatus.
(3)
The information processing apparatus according to (2), wherein the distance sensor detects a distance between a projection display apparatus and a projection area.
(4)
The information processing apparatus according to (3), wherein the distance sensor is provided in the projection display apparatus.
(5)
The information processing apparatus is
The projection control method according to any one of (2) to (3), further comprising: a projection control unit configured to determine a projection area based on an evaluation result by the evaluation unit and to project an image onto the determined projection area. Information processing device.
(6)
The information processing apparatus according to (5), wherein the projection control unit performs control so that projection images are superimposed and projected by a plurality of projection display devices.
(7)
The information processing apparatus according to (6), wherein the evaluation unit evaluates the projection suitability of each projection area in a case where a plurality of projection images are superimposed and projected.
(8)
The projection control unit performs control to overlap and project the plurality of projection images so as to improve the projection compatibility when the projection suitability of the projection area in the case of projecting a single projection image does not satisfy the condition. The information processing apparatus according to (6) or (7).
(9)
The information processing apparatus according to any one of (6) to (8), wherein the projection control unit performs control to overlap and project the entire plurality of projection images.
(10)
The information processing apparatus according to any one of (6) to (8), wherein the projection control unit performs control to perform overlapping projection on a partial region of a projection image.
(11)
The information processing apparatus according to (10), wherein the projection control unit performs projection control so as to overlap a part of areas having the highest priority.
(12)
The information processing apparatus according to any one of (6) to (10), wherein the projection control unit performs control to perform luminance correction of the projection image and to project according to illuminance distribution of a projection area surface.
(13)
The information processing apparatus according to any one of (2) to (12), wherein the evaluation unit evaluates projection compatibility based on the material characteristic of the projection area obtained from the environment sensor.
(14)
The information processing apparatus according to (13), wherein the material property of the projection area includes at least the roughness, smoothness, transmission characteristics, or color tone of the projection area surface.
(15)
The information processing apparatus according to any one of (2) to (14), wherein the evaluation unit evaluates projection suitability based on an environmental characteristic of the projection area obtained from the environmental sensor.
(16)
The information processing apparatus according to (15), wherein the environmental characteristic of the projection area is based on comparison of the illuminance of the projection area surface with the projection illuminance.
(17)
The information processing apparatus according to any one of (2) to (16), wherein the evaluation unit evaluates projection compatibility based on an environment on the projection display apparatus side.
(18)
The information processing apparatus according to (17), wherein the environment on the side of the projection display apparatus includes at least a distance between a projection unit and a projection area, projection brightness, or color information of a projection image.
(19)
Processor is
Recognizing a spatial layout based on data from an environmental sensor,
Evaluating the projection suitability of the projection area in space based on the recognized spatial layout and data from the environmental sensor;
Information processing methods, including:
(20)
Computer,
A recognition unit that recognizes a spatial layout based on data from an environmental sensor;
An evaluation unit that evaluates projection suitability of a projection area in space based on the recognized spatial layout and data from the environment sensor;
Program to function as.

1,

1a Projection system

10, 10a

Information processing apparatus

11, 11a

Projection display apparatus

12, 12a Environment measurement apparatus 100 Control unit 101 Space recognition unit 102 Projection suitability evaluation unit 103 Projection area determination unit 104 Correction unit 105 Projection control unit 110 Communication Part 120 Input part 130 Output part 140 Storage part

Claims

A recognition unit that recognizes a spatial layout based on data from an environmental sensor;
An evaluation unit that evaluates projection suitability of a projection area in space based on the recognized spatial layout and data from the environment sensor;
Equipped with
Information processing device.
The environmental sensor includes one or more imaging sensors and a distance sensor.
The information processing apparatus according to claim 1, wherein the recognition unit recognizes the space layout by performing object recognition based on the image information acquired by the imaging sensor and the distance information acquired by the distance sensor. .
The information processing apparatus according to claim 2, wherein the distance sensor detects a distance between a projection display apparatus and a projection area.
The information processing apparatus according to claim 3, wherein the distance sensor is provided in the projection display apparatus.
The information processing apparatus is
The information processing apparatus according to claim 2, further comprising: a projection control unit configured to determine a projection area based on an evaluation result by the evaluation unit and to project an image on the determined projection area.
The information processing apparatus according to claim 5, wherein the projection control unit performs control to overlap and project projected images by a plurality of projection display devices.
The information processing apparatus according to claim 6, wherein the evaluation unit evaluates the projection suitability of each projection area in a case where a plurality of projection images are superimposed and projected.
The projection control unit performs control to overlap and project the plurality of projection images so as to improve the projection compatibility when the projection suitability of the projection area in the case of projecting a single projection image does not satisfy the condition. The information processing apparatus according to claim 6.
The information processing apparatus according to claim 6, wherein the projection control unit performs control to overlap and project the plurality of projection images entirely.
The information processing apparatus according to claim 6, wherein the projection control unit performs control to perform overlapping projection on a partial area of a projection image.
The information processing apparatus according to claim 10, wherein the projection control unit performs projection control so as to overlap a part of areas having the highest priority.
The information processing apparatus according to claim 6, wherein the projection control unit performs control so that luminance correction of the projection image is performed according to illuminance distribution of a projection area surface and the projection is projected.
The information processing apparatus according to claim 2, wherein the evaluation unit evaluates projection compatibility based on material characteristics of the projection area obtained from the environment sensor.
The information processing apparatus according to claim 13, wherein the material property of the projection area includes at least the roughness, smoothness, transmission property, or color tone of the projection area surface.
The information processing apparatus according to claim 2, wherein the evaluation unit evaluates projection compatibility based on an environmental characteristic of the projection area obtained from the environmental sensor.
The information processing apparatus according to claim 15, wherein the environmental characteristic of the projection area is based on a comparison of the illuminance of the projection area surface and the projection illuminance.
The information processing apparatus according to claim 2, wherein the evaluation unit evaluates projection compatibility based on an environment on a projection display device side.
The information processing apparatus according to claim 17, wherein the environment on the side of the projection display apparatus includes at least a distance between a projection unit and a projection area, projection luminance, or color information of a projection image.
Processor is
Recognizing a spatial layout based on data from an environmental sensor,
Evaluating the projection suitability of the projection area in space based on the recognized spatial layout and data from the environmental sensor;
Information processing methods, including:
Computer,
A recognition unit that recognizes a spatial layout based on data from an environmental sensor;
An evaluation unit that evaluates projection suitability of a projection area in space based on the recognized spatial layout and data from the environment sensor;
Program to function as.